There are enough day-to-day tasks to keep up with, from checking every system is running to something as basic as checking emails. It all takes up time from your day and for a VMware administrator responsible for managing, tracking, and executing VMware tasks and hardware upgrades it can seem like a daunting and time-consuming series of tasks. Not to mention responding to high virtual hardware utilization alerts, provisioning a new VM upon request, or generating reports of aged snapshots require even more human effort. And while it’s working for now, these man-driven resolutions end up taking up way more time and resources in the long run as organizations upscale their VMware Environments.
So what’s the best way to reduce the manual effort associated with the demand and free up administrators’ time? Using ServiceNow Flow Designer with VMware’s PowerCLI PowerShell Module is definitely the place to start.
When you leverage PowerCLI within ServiceNow, all that time spent managing and reporting VMware environments throughout their life cycles gets cut almost in half. But the most important part of all, the results provide an accelerated provisioning and configurations that free up administrators and engineers time. Some of these changes can even be made with little to no interruption to end users of that machine.
PowerCLI is a command-line and scripting tool built on Windows PowerShell, and it provides more than 800 cmdlets for managing and automatic VMware vSphere, VMware Cloud Services, VMware Horizon, and several other VMware environments.
Now let’s jump into an all too familiar use case - responding to high memory utilization alerts for a VM in ServiceNow.
These alerts usually lead to an incident ticket assigned to a VMware engineer which requires them to manually resolve the issue either by investigation or hardware upgrade. To add to that, it can take days if not weeks to schedule an after-hours maintenance window with minimal user impact and then wait for all the necessary approvals on your change request. On top of all of that, doing the actual hardware upgrade during your after hours maintenance window, when you could be eating dinner with your family or watching Netflix.
Before you get ahead of yourself thinking “Great! I’ll just run this PowerCLI script to upgrade memory for every alert that comes in,” we need to define some limits and thresholds so that we can prevent our precious and costly resources from being wasted or over-allocated. For example, I’d check for the following items within my Flow. Any cases where these limits are exceeded should be deferred to manual alert remediation.
- Check recent change requests for the VM configuration item. Has a similar hardware upgrade been performed in the last X days? This could indicate a larger issue and we wouldn’t want to repeatedly upgrade the same resources.
- Calculate the amount of memory to add to the VM and confirm it does not exceed the max limit defined (ex. Add ~20% to current size in increments of 4. The new total size must not exceed 200 GB of memory)
- Confirm new memory size will not exceed the total threshold (N+1) of it’s host/clusters
Now for the fun part - Upgrading a VM’s Memory via PowerCLI
Below is an example script to be used in the powershell step of a custom action and called by our main Flow. We’ll assume that the variables $vCenter, $vmName, $newMemorySize, and $cred (credential alias for service account with appropriate VMware privileges) are inputted via the action/step inputs from the Flow.
First, we’ll connect to that VM’s vCenter and retrieve the VM which is stored in a variable
Now we need to check if Memory HotAdd is enabled on our VM. We can find this information in the VM’s extension data. If it’s not enabled and the VM is powered off, we can go ahead and enable it. Otherwise, we’ll need to throw an error and terminate the automation since we won’t be able to go further.
**Hot Add allows memory (or other resources) to be added to a powered on VM which reduces interruption to the end user. If hot add is not enabled, it can only be enabled if the machine is powered off
Finally, we’ll upgrade the VM’s memory to its new size and then disconnect from our session
*Following this powershell step in your action, don’t forget to validate any error output and create a failure path.
Below is a high level example of the automation from alert creation to resolution: