Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Powershell: Run multiple jobs in parralel and view streaming results from background jobs

Overview

Looking to call a Powershell script that takes in an argument, runs each job in the background, and shows me the verbose output.

Problem I am running into

The script appears to run, but I want to verify this for sure by streaming the results of the background jobs as they are running.

Code

###StartServerUpdates.ps1 Script###

#get list of servers to update from text file and store in array
$servers=get-content c:\serverstoupdate.txt

#run all jobs, using multi-threading, in background
ForEach($server in $servers){
  Start-Job -FilePath c:\cefcu_it\psscripts\PSPatch.ps1 -ArgumentList $server
}

#Wait for all jobs
Get-Job | Wait-Job

#Get all job results
Get-Job | Receive-Job

What I am currently seeing:

Id              Name            State      HasMoreData     Location             Command                  
--              ----            -----      -----------     --------             -------                  
23              Job23           Running    True            localhost            #patch server ...        
25              Job25           Running    True            localhost            #patch server ...        

What I want to see:

Searching for approved updates ...

Update Found:  Security Update for Windows Server 2003 (KB2807986)
Update Found:  Windows Malicious Software Removal Tool - March 2013 (KB890830)

Download complete.  Installing updates ...

The system must be rebooted to complete installation.
cscript exited on "myServer" with error code 3.
Reboot required...
Waiting for server to reboot (35)

Searching for approved updates ...

There are no updates to install.
cscript exited on "myServer" with error code 2.
Servername "myServer" is fully patched after 2 loops

I want to be able to see the output or store that somewhere so I can refer back to be sure the script ran and see which servers rebooted, etc.

Conclusion:

In the past, I ran the script and it went through updating the servers one at a time and gave me the output I wanted, but when I started doing more servers - this task took too long, which is why I am trying to use background jobs with "Start-Job".

Can anyone help me figure this out, please?

like image 610
talbert.houle Avatar asked Mar 22 '13 21:03

talbert.houle


People also ask

How do I run multiple PowerShell scripts at the same time?

Use CTRL+T to create a new powershell instance (a tab is created, which is called powershell 2, I believe) inside Powershell ISE. From the new Powershell tab you can now open a second powershell script and run it aside the script running in other powershell tabs.

What is AsJob in PowerShell?

Automatic Jobs with -AsJob When the job completes, get the result of the job with Receive-Job . Note. Receive-Job returns the result from the cmdlet as if the -AsJob flag were not present. For example, the Receive-Job result of Do-Action -AsJob is of the same type as the result of Do-Action . Azure PowerShell Copy.

Is PowerShell multithreaded?

The default PowerShell session is single-threaded. It runs one command and when it finishes, it moves to the next command. This is nice as it keeps everything repeatable and does not use many resources.

How do I run a PowerShell script in the background?

To run commands in the background in the PowerShell, you need to use Background job cmdlets. Background job means running commands/job in the background without occupying the console.


1 Answers

You may take a look at the module SplitPipeline. It it specifically designed for such tasks. The working demo code is:

# import the module (not necessary in PS V3)
Import-Module SplitPipeline

# some servers (from 1 to 10 for the test)
$servers = 1..10

# process servers by parallel pipelines and output results immediately
$servers | Split-Pipeline {process{"processing server $_"; sleep 1}} -Load 1, 1

For your task replace "processing server $_"; sleep 1 (simulates a slow job) with a call to your script and use the variable $_ as input, the current server.

If each job is not processor intensive then increase the parameter Count (the default is processor count) in order to improve performance.

like image 181
Roman Kuzmin Avatar answered Oct 10 '22 06:10

Roman Kuzmin