Using Powershell To Distribute Script Level Jobs On Remote Servers

Question

More of a theory question...

I have a powershell script that exists on three servers. In this example the three servers are:

1. server1
2. server2
3. server3

I am using another machine, server4, to call script C:\ExampleScript.ps1 remotely using Invoke-Command while specifying the remote machine via the ComputerName parameter. The ultimate goal of the script is to detect whether powershell is running, if it is not, then the computer is "not busy" and can open up the script being called remotely. If the computer is "busy", move onto the next server and continue on through the three machines until all the parameter values have been exhausted. If all machines are busy, it would be ideal if there was a way to periodically check the processes and see if they are still open. In this way, execution of the script can be balanced across the various machines, in an albeit primitive fashion.

Consider the following code:

$servers = "server1","server2","server3"
$data = "param1", "param2", "param3", "param4", "param5", "param6"

#somehow loop through the different servers/data using the above arrays
$job = Invoke-Command $servers[0] {
$ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
if($ProcessActive -eq $null)
{
    "Running"
    Invoke-Command -ComputerName $env:computername -FilePath C:\ExampleScript.ps1 -ArgumentList $data[0]
}
else
{
     "Busy go to next machine"
}
 } -AsJob 
Wait-Job $job 
$r = Receive-Job $job 
$r 

The expected result trying to be achieved is attempting to load balance the script across the machines based on whether there is an active powershell process, if not move onto the next machine and perform the same test and subsequent possible execution. The script should go through all the values as specified in the $data array (or whatever).

Answer 1

I found this question interesting, so I wanted to give it a try.

$servers = "server1","server2","server3"

$data = New-Object System.Collections.ArrayList
$data.AddRange(@("param1", "param2", "param3", "param4", "param5", "param6"))

$jobs = New-Object System.Collections.ArrayList

do 
{
    Write-Host "Checking job states." -ForegroundColor Yellow
    $toremove = @()
    foreach ($job in $jobs)
    {
        if ($job.State -ne "Running")
        {
            $result = Receive-Job $job

            if ($result -ne "ScriptRan")
            {
                Write-Host "  Adding data back to que >> $($job.InData)" -ForegroundColor Green
                $data.Add($job.InData) | Out-Null
            }

            $toremove += $job
        }
    }

    Write-Host "Removing completed/failed jobs" -ForegroundColor Yellow
    foreach ($job in $toremove)
    {
        Write-Host "  Removing job >> $($job.Location)" -ForegroundColor Green
        $jobs.Remove($job) | Out-Null
    }

    # Check if there is room to start another job
    if ($jobs.Count -lt $servers.Count -and $data.Count -gt 0)
    {
        Write-Host "Checking servers if they can start a new job." -ForegroundColor Yellow
        foreach ($server in $servers)
        {

            $job = $jobs | ? Location -eq $server
            if ($job -eq $null)
            {
                Write-Host "  Adding job for $server >> $($data[0])" -ForegroundColor Green
                # No active job was found for the server, so add new job
                $job = Invoke-Command $server -ScriptBlock {
                    param($data, $hostname)
                    $ProcessActive = Get-Process powershell -ErrorAction SilentlyContinue
                    if($ProcessActive -eq $null)
                    {
                        # This will block the thread on the server, so the JobState will not change till it's done or fails.
                        Invoke-Command -ComputerName $hostname -FilePath C:\ExampleScript.ps1 -ArgumentList $data
                        Write-Output "ScriptRan"
                    }
                } -ArgumentList $data[0], $env:computername -AsJob
                $job | Add-Member -MemberType NoteProperty -Name InData -Value $data[0]
                $jobs.Add($job) | Out-Null
                $data.Remove($data[0])
            }
        }
    }
    # Just a manual check of $jobs
    Write-Output $jobs
    # Wait a bit before checking again
    Start-Sleep -Seconds 10
} while ($data.Count -gt 0)

Basically I create an array, and keep it constantly populated with one job for each server.

Data is removed from the list when a new job starts, and is added back if a job fails. This is to avoid servers running the script with the same data/params.

I lack a proper environment to test this properly at the moment, but will give it a whirl at work tomorrow and update my answer with any changes if needed.