How to make PowerCLI script act (almost) like a Windows service.

In the final stage of my preparation to VCAP5-DCA exam (that I passed on September 1st – yeah I couldn’t help to brag a little about it 😉 ) I started to experience issues with SSO in my home lab.
Intermittently and for no obvious reasons SSO was refusing to cooperate and I wasn’t able to authenticate against vCenter. All I was left with was the following error message.

sso_error

The fast and dirty workaround was to restart SSO (Windows) service, but you’d all agree that’s not a professional approach. My problem was – I was really getting out of time before the exam, also my home lab was really resource constrained (and I had a suspicion that these problems are caused by too small amount of RAM I’d assigned to SSO (Windows) server). I didn’t want to “waste time” for proper troubleshooting and I was afraid that the only solution would be to increase the RAM for SSO virtual machine (which was simply impossible), so I decided to write a short PowerCLI script that would periodically check the SSO status (by simply attempting to connect to my vCenter) and restart the SSO service if needed.

Before I show you this script, let me stress out that SSO is way too important component of VMware Infrastructure to accept even intermittent problems like the one I just described. If you’re having any issues with SSO (especially outside your home lab), you should troubleshoot it properly, preferably with assistance of VMware support. In fact I am not recommending scripted solutions of this kind (check service status, restart if needed) for anything outside home lab and for any period of time longer than few days before your exam ;).

This script starts “infinite” loop in Line 101, then inside the loop tries to connect to pre-definied ($vcenter_srv variable) vCenter server every 10 to 12 minutes (I wanted to randomize the interval between attempts somehow, just in case the problem was one of these “self-restarting service” issues). What happens next is not much more than checking $error variable and writing it down in the log. After first failed attempt the script takes no action – it just waits another minute. If 2nd attempt is unsuccessful the script tries to check the status of SSO service (aka ssotomcat).
Please note this step can fail too – and if you’re running this script (like you should) on server local to SSO it means you’re in serious trouble at the Windows authentication level.
Depending on the status retrieved, the script will attempt to restart the SSO (if it is running), start it (if it is stopped) or again do nothing if the service is in one of the transition states like “Pending start” etc.
Because the script is intended to run for prolonged periods of time it also rotates its log every 24 hours, here I decided to use internal counter ($script_uptime) rather than invoke get-data after every iteration… just because I could ;).

OK – so at this point we have a script that will run until we kill its PowerCLI session, but it wasn’t enough for me.
I just didn’t want to “waste time” to log on to my vCenter/SSO server just to start this script every time I boot my lab, nor did I want to waste my precious RAM to leave RDP session opened there (for the script to survive).

Luckily Windows Task Scheduler offers an opportunity to trigger a task “At system startup” and making a task out of PowerCLI script is well documented process (that I will show you anyway in the few screenshots to come).

First of all – we need to make sure our task is running under the user (or better yet – service) account that has “Log on as batch job privilege and is allowed to log on to vCenter server.
Let’s also ensure the task can be run whether that account is actually logged on or not.

sso_watchdog_01

Secondly, we configure our task to be triggered at system startup (don’t forget to enable this trigger!)

sso_watchdog_02

Now, our action will be of course to start a program and this program will be PowerShell interpreter, then we will pass our script as argument, so the “Edit Action” window could look somewhat like that:

sso_watchdog_03

That’s not really readable, is it?

Well “Program / script:” textbox should contain something like that:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe.
(path to where your powershell.exe is installed), while “Add arguments (optional):” textbox should contain path where you have your PowerCLI snap-in installed (together with path to the script itself):
-PSconsolefile “d:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\vim.psc1” “& ‘d:\tools\scripts\sso_watchdog.ps1′”.

For your convenience I put both examples above in the script code itself.

At this point only small adjustments are needed, I prefer to start this task only if some network connection is available.

sso_watchdog_04

And make sure I can start my task manually.

sso_watchdog_05

This setup pretty much did the trick for me – my script was starting automatically every time I booted my lab, I had quite verbose log available for me to check at any given moment and I was sure that simple recovery actions (SSO restart) will happen w/o any manual action.
Of course the behavior of this script can’t be controlled via Services console, nor will it write entries to Windows Event Log, but it can be started and stopped via Task Scheduler console and has its own (text-based) log file that rotates every 24 hours. That’s good enough in my book to call it “almost service” :).

In this post I used automated restarts of failing SSO only as an example (not even very good one) how to make your PowerCLI script work in unattended mode. In fact you can use the “Task Scheduler trick” I described for any PowerCLI script that you want to be started together with your Windows server and work without your manual intervention.

I hope you will find this post useful, if so – feel free to share it.
As always – any constructive feedback is welcome!

Sebastian Baryło

Successfully jumping to conclusions since 2001.