How to capture process output asynchronously in powershell?

Question

I want to capture stdout and stderr from a process that I start in a Powershell script and display it asynchronously to the console. I've found some documentation on doing this through MSDN and other blogs.

After creating and running the example below, I can't seem to get any output to be displayed asynchronously. All of the output is only displayed when the process terminates.

$ps = new-object System.Diagnostics.Process
$ps.StartInfo.Filename = "cmd.exe"
$ps.StartInfo.UseShellExecute = $false
$ps.StartInfo.RedirectStandardOutput = $true
$ps.StartInfo.Arguments = "/c echo `"hi`" `& timeout 5"

$action = { Write-Host $EventArgs.Data  }
Register-ObjectEvent -InputObject $ps -EventName OutputDataReceived -Action $action | Out-Null

$ps.start() | Out-Null
$ps.BeginOutputReadLine()
$ps.WaitForExit()

In this example, I was expecting to see the output of "hi" on the commandline before the end of program execution because the OutputDataReceived event should have been triggered.

I've tried this using other executables - java.exe, git.exe, etc. All of them have the same effect, so I'm left to think that there is something simple that I'm not understanding or have missed. What else needs to be done to read stdout asynchronously?

Answer 1

Unfortunately asynchronous reading is not that easy if you want to do it properly. If you call WaitForExit() without timeout you could use something like this function I wrote (based on C# code):

function Invoke-Executable {
    # Runs the specified executable and captures its exit code, stdout
    # and stderr.
    # Returns: custom object.
    param(
        [Parameter(Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [String]$sExeFile,
        [Parameter(Mandatory=$false)]
        [String[]]$cArgs,
        [Parameter(Mandatory=$false)]
        [String]$sVerb
    )

    # Setting process invocation parameters.
    $oPsi = New-Object -TypeName System.Diagnostics.ProcessStartInfo
    $oPsi.CreateNoWindow = $true
    $oPsi.UseShellExecute = $false
    $oPsi.RedirectStandardOutput = $true
    $oPsi.RedirectStandardError = $true
    $oPsi.FileName = $sExeFile
    if (! [String]::IsNullOrEmpty($cArgs)) {
        $oPsi.Arguments = $cArgs
    }
    if (! [String]::IsNullOrEmpty($sVerb)) {
        $oPsi.Verb = $sVerb
    }

    # Creating process object.
    $oProcess = New-Object -TypeName System.Diagnostics.Process
    $oProcess.StartInfo = $oPsi

    # Creating string builders to store stdout and stderr.
    $oStdOutBuilder = New-Object -TypeName System.Text.StringBuilder
    $oStdErrBuilder = New-Object -TypeName System.Text.StringBuilder

    # Adding event handers for stdout and stderr.
    $sScripBlock = {
        if (! [String]::IsNullOrEmpty($EventArgs.Data)) {
            $Event.MessageData.AppendLine($EventArgs.Data)
        }
    }
    $oStdOutEvent = Register-ObjectEvent -InputObject $oProcess `
        -Action $sScripBlock -EventName 'OutputDataReceived' `
        -MessageData $oStdOutBuilder
    $oStdErrEvent = Register-ObjectEvent -InputObject $oProcess `
        -Action $sScripBlock -EventName 'ErrorDataReceived' `
        -MessageData $oStdErrBuilder

    # Starting process.
    [Void]$oProcess.Start()
    $oProcess.BeginOutputReadLine()
    $oProcess.BeginErrorReadLine()
    [Void]$oProcess.WaitForExit()

    # Unregistering events to retrieve process output.
    Unregister-Event -SourceIdentifier $oStdOutEvent.Name
    Unregister-Event -SourceIdentifier $oStdErrEvent.Name

    $oResult = New-Object -TypeName PSObject -Property ([Ordered]@{
        "ExeFile"  = $sExeFile;
        "Args"     = $cArgs -join " ";
        "ExitCode" = $oProcess.ExitCode;
        "StdOut"   = $oStdOutBuilder.ToString().Trim();
        "StdErr"   = $oStdErrBuilder.ToString().Trim()
    })

    return $oResult
}

It captures stdout, stderr and exit code. Example usage:

$oResult = Invoke-Executable -sExeFile 'ping.exe' -cArgs @('8.8.8.8', '-a')
$oResult | Format-List -Force 

For more info and alternative implementations (in C#) read this blog post.

Answer 2

Based on Alexander Obersht's answer I've created a function that uses timeout and asynchronous Task classes instead of event handlers. According to Mike Adelson

Unfortunately, this method(event handlers) provides no way to know when the last bit of data has been received. Because everything is asynchronous, it is possible (and I have observed this) for events to fire after WaitForExit() has returned.

function Invoke-Executable {
# from https://stackoverflow.com/a/24371479/52277
    # Runs the specified executable and captures its exit code, stdout
    # and stderr.
    # Returns: custom object.
# from http://www.codeducky.org/process-handling-net/ added timeout, using tasks
param(
        [Parameter(Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [String]$sExeFile,
        [Parameter(Mandatory=$false)]
        [String[]]$cArgs,
        [Parameter(Mandatory=$false)]
        [String]$sVerb,
        [Parameter(Mandatory=$false)]
        [Int]$TimeoutMilliseconds=1800000 #30min
    )
    Write-Host $sExeFile $cArgs

    # Setting process invocation parameters.
    $oPsi = New-Object -TypeName System.Diagnostics.ProcessStartInfo
    $oPsi.CreateNoWindow = $true
    $oPsi.UseShellExecute = $false
    $oPsi.RedirectStandardOutput = $true
    $oPsi.RedirectStandardError = $true
    $oPsi.FileName = $sExeFile
    if (! [String]::IsNullOrEmpty($cArgs)) {
        $oPsi.Arguments = $cArgs
    }
    if (! [String]::IsNullOrEmpty($sVerb)) {
        $oPsi.Verb = $sVerb
    }

    # Creating process object.
    $oProcess = New-Object -TypeName System.Diagnostics.Process
    $oProcess.StartInfo = $oPsi


    # Starting process.
    [Void]$oProcess.Start()
# Tasks used based on http://www.codeducky.org/process-handling-net/    
 $outTask = $oProcess.StandardOutput.ReadToEndAsync();
 $errTask = $oProcess.StandardError.ReadToEndAsync();
 $bRet=$oProcess.WaitForExit($TimeoutMilliseconds)
    if (-Not $bRet)
    {
     $oProcess.Kill();
    #  throw [System.TimeoutException] ($sExeFile + " was killed due to timeout after " + ($TimeoutMilliseconds/1000) + " sec ") 
    }
    $outText = $outTask.Result;
    $errText = $errTask.Result;
    if (-Not $bRet)
    {
        $errText =$errText + ($sExeFile + " was killed due to timeout after " + ($TimeoutMilliseconds/1000) + " sec ") 
    }
    $oResult = New-Object -TypeName PSObject -Property ([Ordered]@{
        "ExeFile"  = $sExeFile;
        "Args"     = $cArgs -join " ";
        "ExitCode" = $oProcess.ExitCode;
        "StdOut"   = $outText;
        "StdErr"   = $errText
    })

    return $oResult
}

Answer 3

I couldn't get either of these examples to work with PS 4.0.

I wanted to run puppet apply from an Octopus Deploy package (via Deploy.ps1) and see the output in "real time" rather than wait for the process to finish (an hour later), so I came up with the following:

# Deploy.ps1

$procTools = @"

using System;
using System.Diagnostics;

namespace Proc.Tools
{
  public static class exec
  {
    public static int runCommand(string executable, string args = "", string cwd = "", string verb = "runas") {

      //* Create your Process
      Process process = new Process();
      process.StartInfo.FileName = executable;
      process.StartInfo.UseShellExecute = false;
      process.StartInfo.CreateNoWindow = true;
      process.StartInfo.RedirectStandardOutput = true;
      process.StartInfo.RedirectStandardError = true;

      //* Optional process configuration
      if (!String.IsNullOrEmpty(args)) { process.StartInfo.Arguments = args; }
      if (!String.IsNullOrEmpty(cwd)) { process.StartInfo.WorkingDirectory = cwd; }
      if (!String.IsNullOrEmpty(verb)) { process.StartInfo.Verb = verb; }

      //* Set your output and error (asynchronous) handlers
      process.OutputDataReceived += new DataReceivedEventHandler(OutputHandler);
      process.ErrorDataReceived += new DataReceivedEventHandler(OutputHandler);

      //* Start process and handlers
      process.Start();
      process.BeginOutputReadLine();
      process.BeginErrorReadLine();
      process.WaitForExit();

      //* Return the commands exit code
      return process.ExitCode;
    }
    public static void OutputHandler(object sendingProcess, DataReceivedEventArgs outLine) {
      //* Do your stuff with the output (write to console/log/StringBuilder)
      Console.WriteLine(outLine.Data);
    }
  }
}
"@

Add-Type -TypeDefinition $procTools -Language CSharp

$puppetApplyRc = [Proc.Tools.exec]::runCommand("ruby", "-S -- puppet apply --test --color false ./manifests/site.pp", "C:\ProgramData\PuppetLabs\code\environments\production");

if ( $puppetApplyRc -eq 0 ) {
  Write-Host "The run succeeded with no changes or failures; the system was already in the desired state."
} elseif ( $puppetApplyRc -eq 1 ) {
  throw "The run failed; halt"
} elseif ( $puppetApplyRc -eq 2) {
  Write-Host "The run succeeded, and some resources were changed."
} elseif ( $puppetApplyRc -eq 4 ) {
  Write-Warning "WARNING: The run succeeded, and some resources failed."
} elseif ( $puppetApplyRc -eq 6 ) {
  Write-Warning "WARNING: The run succeeded, and included both changes and failures."
} else {
  throw "Un-recognised return code RC: $puppetApplyRc"
}

Credit goes to T30 and Stefan Goßner