During the automation/SMA project I’m currently doing we started seeing some very strange behaviour when we started linking the bits (runbooks) together to test the overall orchestration. I’ll try and simplify – consider these two workflows:
Worfklow DeployStuff
{
Param ($server1, $server2)
DeployServer -servername $server1
DeployServer -servername $server2
}
Workflow DeployServer
{
Param ($ServerName)
InlineScript
{
$ErrorActionPreference = "Stop"
$servername = $using:servername
New-Adcomputer -computername $servername -ou "OU=SomeRighteousOu,DC=whatever,DC=com"
Write-verbose "Creating computer account for computer $servername"
} -PSComputerName "Domaincontroller.domain.com"
}
Mind you, these are somewhat mega-simplified versions of our production runbooks, but for the sake of readability I wanted to keep it simple.
The code shows two runbooks/workflows, one calling the other. The first one takes two servernames as parameters (these are actually Hyper-V replica “pairs” to be deployed simultaneously). The master workflow kicks of the “child” workflow twice, once for each computername.
What we started seeing was that when a child workflow is kicked off twice in rapid succession like this, the second “run” starts messing up the “$using:” parameters. It would actually try and create the same (first) computer account twice. With some verbose logging we could actually see that everything was well outside of the inlinescript. After some digging, my working hypothesis is that this stuff breaks down when the following is true:
1. The InlineScript uses $ErrorActionPreference
2. The two instances of the same workflow are kicked off within seconds of each other. The sweet spot seemed to be around 2 minutes. That is, if I added a “start-sleep -seconds 120″ in between the two “DeployServer” calls, it would succeed.
3. The InlineScript uses the “PSComputerName” parameter to kick off the script on another computer than where the workflow is running.
I’m using the word “workflow” deliberately here, as I was able to reproduce the problem using worklows on my workstation, so it has nothing to do with SMA.
After some reflection (and an ice cream) I decided that there must be some bug in how Inlinescripts reuse remoting sessions when the PSComputerName parameter is used to invoke a scriptblock on a remote machine. So, I went back to the drawing board and ran this simple workflow in my test lab on Microsoft Azure:
workflow ConfigureAMachine
{
Param ($ServiceName)
InlineScript
{
$VerbosePreference = "Continue"
Import-Module servermanager
$servicename = $using:servicename
$service = get-service -Name $servicename
Write-Verbose $service.Status
} -PSComputerName "TrondCloudVM01" #-PSDisableSerialization $true
}
ConfigureAMachine -ServiceName "spooler"
As you can see, I added some verbosing. The first time I run the code, I get a ton of verbose info:
VERBOSE: [TrondCloudVM01]:Loading module from path ‘C:\Windows\system32\WindowsPowerShell\v1.0\Modules\servermanager\servermanager.psd1′.
VERBOSE: [TrondCloudVM01]:Loading ‘FormatsToProcess’ from path ‘C:\Windows\system32\WindowsPowerShell\v1.0\Modules\servermanager\Feature.format.ps1xml’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Get-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Install-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Uninstall-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Loading module from path ‘C:\Windows\system32\WindowsPowerShell\v1.0\Modules\servermanager\ServerManager.psm1′.
VERBOSE: [TrondCloudVM01]:Exporting cmdlet ‘Get-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Exporting cmdlet ‘Install-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Exporting cmdlet ‘Uninstall-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Exporting alias ‘Add-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Exporting alias ‘Remove-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Exporting function ‘Enable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Exporting function ‘Disable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Get-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Install-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Uninstall-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing function ‘Disable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Importing function ‘Enable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Importing alias ‘Add-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing alias ‘Remove-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Running
Then, if I run it again a few seconds later I get this:
VERBOSE: [TrondCloudVM01]:Loading module from path ‘C:\Windows\system32\WindowsPowerShell\v1.0\Modules\servermanager\servermanager.psd1′.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Get-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Install-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing cmdlet ‘Uninstall-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing function ‘Disable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Importing function ‘Enable-ServerManagerStandardUserRemoting’.
VERBOSE: [TrondCloudVM01]:Importing alias ‘Add-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Importing alias ‘Remove-WindowsFeature’.
VERBOSE: [TrondCloudVM01]:Running
That’s a LOT less verbose stuff. I’m assuming that the session opened in the first run lingers a bit, which allows the second run to reuse it. Which I’m all for of course – as long as it doesn’t ruin anything for me (which it kinda does).
One a side note, this “session lingering” is a stark contrast to what happens when you simply run an invoke-command against a remote computer. When you use invoke-command, the session is immediately closed after the command has ben run. Using Inlinescripts on the other hand, it is actually possible to see the lingering session using “Get-PsSession -ComputerName <targetcomputer>”:
Write-verbose "Before WF Run" Get-PSSession -ComputerName "TrondCloudVM01" Write-verbose "WF Run" ConfigureAMachine -ServiceName "spooler" -Verbose Write-verbose "After WF Run" Get-PSSession -ComputerName "TrondCloudVM01"
Here I check to see whether there are any sessions opened on my target computer (which is getting pretty sick of the get-service cmdlet at this point) before and after I run my test workflow. Here are the results:
As you can see, no sessions before the workflow ran, and a disconnected session afterwards. If I perform the same test once more immediately after, I seem to be reusing the disconnected session, as the verbose output is less chatty:
Just to clarify: This reusing of the opened session is not a problem by itself. The fact that “$using:” doesn’t seem to work correctly when the WF reuses an existing session is the problem.
I’ll continue to investigate. In the meantime I would recommend that you use “Invoke-Command” instead of “InlineScript” when doing workflow-based remoting – even though it involves those pesky “argumentlist” parameters I thought we were done with.
