PowerShell - redirect executable's stderr to file or variable but still have stdout go to console

Question

I'm writing a script to download several repositories from GitHub. Here is the command to download a repository:

git clone "$RepositoryUrl" "$localRepoDirectory"

When I run this command it displays some nice progress information in the console window that I want displayed.

The problem is that I also want to be able to detect if any errors have occurred while downloading. I found this post that talks about redirecting the various streams, so I tried:

(git clone "$RepositoryUrl" "$localRepoDirectory") 2> $errorLogFilePath

This pipes any errors from stderr to my file, but no longer displays the nice progress information in the console.

I can use the Tee-Object like so:

(git clone "$RepositoryUrl" "$localRepoDirectory") | Tee-Object -FilePath $errorLogFilePath

and I still get the nice progress output, but this pipes stdout to the file, not stderr; I'm only concerned with detecting errors.

Is there a way that I can store any errors that occur in a file or (preferably) a variable, while also still having the progress information piped to the console window? I have a feeling that the answer might lie in redirecting various streams into other streams as discusses in this post, but I'm not really sure.

======== Update =======

I'm not sure if the git.exe is different than your typical executable, but I've done some more testing and here is what I've found:

$output = (git clone "$RepositoryUrl" "$localRepoDirectory")

$output always contains the text "Cloning into '[localRepoDirectory]'...", whether the command completed successfully or produced an error. Also, the progress information is still written to the console when doing this. This leads me to think that the progress information is not written via stdout, but by some other stream?

If an error occurs the error is written to the console, but in the usual white foreground color, not the typical red for errors and yellow for warnings. When this is called from within a cmdlet function and the command fails with an error, the error is NOT returned via the function's -ErrorVariable (or -WarningVariable) parameter (however if I do my own Write-Error that does get returned via -ErrorVariable). This leads me to think that git.exe doesn't write to stderr, but when we do:

(git clone "$RepositoryUrl" "$localRepoDirectory") 2> $errorLogFilePath

The error message is written to the file, so that makes me think that it does write to stderr. So now I'm confused...

======== Update 2 =======

So with Byron's help I've tried a couple of more solutions using a new process, but still can't get what I want. When using a new process I never get the nice progress written to the console.

The three new methods that I've tried both use this bit of code in common:

$process = New-Object System.Diagnostics.Process
$process.StartInfo.Arguments = "clone ""$RepositoryUrl"" ""$localRepoDirectory"""
$process.StartInfo.UseShellExecute = $false
$process.StartInfo.RedirectStandardOutput = $true
$process.StartInfo.RedirectStandardError = $true
$process.StartInfo.CreateNoWindow = $true
$process.StartInfo.WorkingDirectory = $WORKING_DIRECTORY
$process.StartInfo.FileName = "git"

Method 1 - Run in new process and read output afterwards:

$process.Start()
$process.WaitForExit()
Write-Host Output - $process.StandardOutput.ReadToEnd()    
Write-Host Errors - $process.StandardError.ReadToEnd()

Method 2 - Get output synchronously:

$process.Start()
while (!$process.HasExited)
{
    Write-Host Output - $process.StandardOutput.ReadToEnd()
    Write-Host Error Output - $process.StandardError.ReadToEnd()

    Start-Sleep -Seconds 1
}

Even though this looks like it would write the output while the process is running, it doesn't write anything until after the process exits.

Method 3 - Get output asynchronously:

Register-ObjectEvent -InputObject $process -EventName "OutputDataReceived" -Action {Write-Host Output Data - $args[1].Data }
Register-ObjectEvent -InputObject $process -EventName "ErrorDataReceived" -Action { Write-Host Error Data - $args[1].Data }
$process.Start()
$process.BeginOutputReadLine()
$process.BeginErrorReadLine()
while (!$process.HasExited)
{
    Start-Sleep -Seconds 1
}

This does output data while the process is working which is good, but it still doesn't display the nice progress information :(

Answer 1

I think I have your answer. I'm working with PowerShell for a while and created several build systems. Sorry if the script is a bit long, but it works.

$dir = <your dir>
$global:log = <your log file which must be in the global scope> # Not global = won't work

function Create-Process {
    $process = New-Object -TypeName System.Diagnostics.Process
    $process.StartInfo.CreateNoWindow = $false
    $process.StartInfo.RedirectStandardError = $true
    $process.StartInfo.UseShellExecute = $false
    return $process
}

function Terminate-Process {
    param([System.Diagnostics.Process]$process)

    $code = $process.ExitCode
    $process.Close()
    $process.Dispose()
    Remove-Variable process
    return $code
}

function Launch-Process {
    param([System.Diagnostics.Process]$process, [string]$log, [int]$timeout = 0)

    $errorjob = Register-ObjectEvent -InputObject $process -EventName ErrorDataReceived -SourceIdentifier Common.LaunchProcess.Error -action {
        if(-not [string]::IsNullOrEmpty($EventArgs.data)) {
            "ERROR - $($EventArgs.data)" | Out-File $log -Encoding ASCII -Append
            Write-Host "ERROR - $($EventArgs.data)"
        }
    }
    $outputjob = Register-ObjectEvent -InputObject $process -EventName OutputDataReceived -SourceIdentifier Common.LaunchProcess.Output -action {
        if(-not [string]::IsNullOrEmpty($EventArgs.data)) {
            "Out - $($EventArgs.data)" | Out-File $log -Encoding ASCII -Append
            Write-Host "Out - $($EventArgs.data)"
        }
    }

    if($errorjob -eq $null) {
        "ERROR - The error job is null" | Out-File $log -Encoding ASCII -Append
        Write-Host "ERROR - The error job is null"
    }

    if($outputjob -eq $null) {
        "ERROR - The output job is null" | Out-File $log -Encoding ASCII -Append
        Write-Host "ERROR - The output job is null"
    }

    $process.Start() 
    $process.BeginErrorReadLine()

    if($process.StartInfo.RedirectStandardOutput) {
        $process.BeginOutputReadLine() 
    }

    $ret = $null
    if($timeout -eq 0)
    {
        $process.WaitForExit()
        $ret = $true
    }
    else
    {
        if(-not($process.WaitForExit($timeout)))
        {
            Write-Host "ERROR - The process is not completed, after the specified timeout: $($timeout)"
            $ret = $false
        }
        else
        {
            $ret = $true
        }
    }

    # Cancel the event registrations
    Remove-Event * -ErrorAction SilentlyContinue
    Unregister-Event -SourceIdentifier Common.LaunchProcess.Error
    Unregister-Event -SourceIdentifier Common.LaunchProcess.Output
    Stop-Job $errorjob.Id
    Remove-Job $errorjob.Id
    Stop-Job $outputjob.Id
    Remove-Job $outputjob.Id

    $ret
}

$repo = <your repo>

$process = Create-Process
$process.StartInfo.RedirectStandardOutput = $true
$process.StartInfo.FileName = "git.exe"
$process.StartInfo.Arguments = "clone $($repo)"
$process.StartInfo.WorkingDirectory = $dir
Launch-Process $process $global:log
Terminate-Process $process

The log file must be in the global scope because the routine which runs the event processing is not in the script scope.

Sample of my log file:

Out - Cloning into ''... ERROR - Checking out files: 22% (666/2971)
ERROR - Checking out files: 23% (684/2971)
ERROR - Checking out files: 24% (714/2971)

Answer 2

You can do this by putting the git clone command inside an advanced function e.g.:

function Clone-Git {
    [CmdletBinding()]
    param($repoUrl, $localRepoDir)

    git clone $repoUrl $localRepoDir
}

Clone-Git $RepositoryUrl $localRepoDirectory -ev cloneErrors

$cloneErrors