Run a commandline process and get the output while that process is still running?

Question

How I can run a commandline process and get the output while that process is still running?

What I mean is to run a CLI process with its own progressbar, the executable itself takes a long time to finish the operation so I want to take that progress information from the own process to display the progress in my application, other way I don't have any info to display a progress until process finishes.

I'm working in a WindowsForm project, not a Console App.

I've tried to do the same using FFMPEG.exe (x64) and I can read the "progress" while FFMPEG is running, I can pick the progress from FFMPEG and do what I want, but with this executable I just can't do it and I don't know if is possible.

The program is "dbPowerAmp CoreConverter", it's a music converter, I think the program sends all the outputs in Unicode encoding because to read the output I need to set the output encoding to Unicode.

...Other problem is I can't find a way to read the StandardError output of this process even using Unicode so please if someone can help me with those two problems.

Here is the application: http://www.dbpoweramp.com/install/dMC-R14.4-Ref-Trial.exe

Here is an example output of the program launched directly from CMD:

(What I need is to pick the progressbar " * " asterisk character amount while process is running to calculate and display that percentage also in my application)

And here is my code:

Private Shared CoreConverter As New Process()

Private Shared CoreConverter_Info As New ProcessStartInfo() With { _
              .CreateNoWindow = True, _
              .UseShellExecute = False, _
              .RedirectStandardOutput = True, _
              .RedirectStandardError = True _
}

Private Shared Sub Run_CoreConverter()


    ' Just for testing CMD Unicode output:
    '
    ' CoreConverter_Info.FileName = "cmd"
    ' CoreConverter_Info.Arguments = "/U /C C:\CoreConverter.exe " & CoreConverter_Info.Arguments


    CoreConverter_Info.FileName = "C:\CoreConverter.exe"
    CoreConverter_Info.Arguments = String.Format("-infile=""{0}"" -outfile=""{1}"" -convert_to=""mp3 (Lame)""" ..., blah blah blah)
    CoreConverter_Info.StandardErrorEncoding = System.Text.Encoding.Unicode
    CoreConverter_Info.StandardOutputEncoding = System.Text.Encoding.Unicode
    CoreConverter.StartInfo = CoreConverter_Info
    CoreConverter.EnableRaisingEvents = True
    CoreConverter.Start()
    CoreConverter.WaitForExit()


    Dim readerStdOut As IO.StreamReader = CoreConverter.StandardOutput

    ' This part works with FFMPEG executable but not with Coreconverter.exe,
    ' What I mean is that the msgbox is displayed when CoreConverter.exe finishes :( 
    ' but with FFMPEG I can read the output while FFMPEG still running.
    While CoreConverter.StandardOutput.EndOfStream = True
        MsgBox(CoreConverter.StandardOutput.ReadLine)
    End While

    If CoreConverter.ExitCode <> 0 Then
        ' Throw New Exception(CoreConverter.StandardError.ReadToEnd)

        ' No way to read the ErrorOutput...
        MessageBox.Show(CoreConverter.StandardError.ReadToEnd, "CoreConverter", MessageBoxButtons.OK, MessageBoxIcon.Error)

        MessageBox.Show(CoreConverter.StandardOutput.ReadToEnd, "CoreConverter", MessageBoxButtons.OK, MessageBoxIcon.Error)
    End If

End Sub

UPDATE:

Really I'm so desesperated, all of my intents are epic fails, I've tried all what my skills can do (but that is not much).

Even using multithread to run a process in a separated thread while I try to get the output from the main thread I can't try to retrieve the process output until the process exited, it's crazy!, I don't understand why this happens.

Just is a crazy thing where I'm loosing a lot of hours by day, some people tell me that perhaps making a COM Object I can do it, well, I don't want to spent months learning how to make a COM Object just to pick a couple of characters from console, That months learning that would not help me with anything more in my life, simply as that, I need something more basic, maybe the idea of reading the buffer console could work, but sure I would not be able to read the buffer until the process exited, so.

So. What can I do with this damn process?

Just to make more than an update of my question to say my own desesperation words I will paste here a multi thread thing I've tried.

...Like I've said I can't try to read the outputs while process still runing, and the error output just can't be readed (string always is empty) even when the process finishes (with an intentioned error ofcourse).

Public Shared error_output As String
Public Shared standard_output As String
Public Shared active As Boolean = False ' CoreConverter Thread is active?

Public Shared MyThread As Threading.Thread = New Threading.Thread(AddressOf Run_CoreConverter)

Public Shared Sub Convert_To_MP3(ByVal In_File As String, _

' blah blah blah
'Run_CoreConverter()

    MyThread = New Threading.Thread(AddressOf Run_CoreConverter)
    ' MyThread.IsBackground = True
    active = True
    MyThread.Start()
    Get_Output()

End Sub

Public Shared Sub Get_Output()

While active ' While Coreconverter.exe is runing...

    Try
        If Not CoreConverter.HasExited Then
            MsgBox(CoreConverter.StandardOutput.ReadToEnd) ' This will not be displayed until process has exited...
        End If
    Catch ex As Exception

    End Try

    If error_output IsNot Nothing Then
        MsgBox(error_output) ' This will not be displayed until process has exited...
    End If

    If standard_output IsNot Nothing Then
        MsgBox(standard_output) ' This will not be displayed until process has exited...
    End If

End While

MsgBox("end active")

End Sub

Public Shared Sub Run_CoreConverter()
    CoreConverter_Info.FileName = CoreConverter_Location
    CoreConverter_Info.StandardErrorEncoding = System.Text.Encoding.Unicode
    CoreConverter_Info.StandardOutputEncoding = System.Text.Encoding.Unicode
    CoreConverter.StartInfo = CoreConverter_Info
    CoreConverter.EnableRaisingEvents = False

    CoreConverter.Start()
    ' CoreConverter.WaitForExit()

    ' Threading.Thread.Sleep(2000)
    ' For x As Integer = 0 To 99999999
    error_output = CoreConverter.StandardError.ReadToEnd
    standard_output = CoreConverter.StandardOutput.ReadToEnd
    ' Next

    If CoreConverter.ExitCode <> 0 Then
        Throw New Exception(CoreConverter.StandardError.ReadToEnd)
         MessageBox.Show(CoreConverter.StandardError.ReadToEnd, "CoreConverter", MessageBoxButtons.OK, MessageBoxIcon.Error)
         MessageBox.Show(CoreConverter.StandardOutput.ReadToEnd, "CoreConverter", MessageBoxButtons.OK, MessageBoxIcon.Error)
    End If

    CoreConverter.Close()
    active = False

End Sub

Answer 1

I think there may be bugs in dbPowerAmp's encoding. The output looks great in a cmd.exe /u Unicode environment, but it seems when you hook up a .Net Process object, it ends up with null bytes in between characters. You can work around this by discarding null bytes in between good UTF8 characters.

I'm more fluent in PowerShell, so here's what I wrote to prove that this will work.

$progressCounter = 0.0
$progressScaleString = "0%-----------25%-----------50%-----------75%-----------100%"
$psi = new-object system.diagnostics.processstartinfo
$psi.FileName = "C:\Program Files (x86)\Illustrate\dBpoweramp\CoreConverter.exe"
# Moby Dick audiobook available at <http://ia600208.us.archive.org/0/items/moby_dick_librivox/mobydick_135_melville.mp3>
$psi.arguments =  '-infile="C:\AudioBooks\mobydick_135_melville.mp3" -outfile="c:\AudioBooks\mobydick_135_melville.flac" -convert_to="flac" -encoding="SLOW"'
$psi.StandardOutputEncoding = [System.Text.Encoding]::Unicode
$psi.RedirectStandardOutput = $true
$psi.UseShellExecute = $false
$proc = new-object System.Diagnostics.Process
$proc.StartInfo = $psi
$proc.Start();
#Look for the magic progress bar string
$outputBuf = ""
while($true){ $chr = [char]$proc.StandardOutput.BaseStream.ReadByte(); if($chr -ne [char]0){ $outputBuf += $chr; } if($outputBuf.Contains($progressScaleString)){ break; }else{ sleep .01; }}
#We've seen the progress ruler, now start counting the pips.
while($progressCounter -le 100.0){ $chr = $proc.StandardOutput.BaseStream.ReadByte(); if($chr -lt 0){break;} if([char]$chr -eq [char]"*"){ $progressCounter += 100.0/($progressScaleString.Length+1); write-host $progressCounter; }}

Here's my crack at porting this to VB; Just run it on a worker thread and hook PercentDone up to your UI:

Public Event PercentDone(ByVal Percent As Float)

Private Shared CoreConverter As New Process()

Private Shared CoreConverter_Info As New ProcessStartInfo() With { _
              .CreateNoWindow = True, _
              .UseShellExecute = False, _
              .RedirectStandardOutput = True, _
              .RedirectStandardError = True _
}

Private Shared Sub Run_CoreConverter()

    Dim ProgressScaleString As String = "0%-----------25%-----------50%-----------75%-----------100%"
    Dim ProgressCounter As Float = 0.0

    CoreConverter_Info.FileName = "C:\CoreConverter.exe"
    CoreConverter_Info.Arguments = String.Format("-infile=""{0}"" -outfile=""{1}"" -convert_to=""mp3 (Lame)""" ..., blah blah blah)
    CoreConverter_Info.StandardErrorEncoding = System.Text.Encoding.Unicode
    CoreConverter_Info.StandardOutputEncoding = System.Text.Encoding.Unicode
    CoreConverter.StartInfo = CoreConverter_Info
    CoreConverter.Start()

    Dim OutputBuf As String = ""
    Dim chr As Byte;
    While True
      chr = CoreConverter.StandardOutput.BaseStream.ReadByte();
      If chr < 0 Then
        Exit While
      ElseIf chr <> 0
        OutputBuf += CType(chr, Char)
      End If
      If OutputBuf.Contains(ProgressScaleString) Then
        Exit While
      Else
        System.Threading.Thread.Sleep(10)
      End If
    End While

    While ProgressCounter <= 100
      chr = CoreConverter.StandardOutput.BaseStream.ReadByte()
      If chr <= 0 Then
        Exit While
      End If
      If chr == CType("*"C, Char) Then
        ProgressCounter += 100.0/($progressScaleString.Length+1)
        RaiseEvent PercentDone(ProgressCounter)
      End If
    End While

End Sub

Answer 2

The process is writing a line of stars (*). I is possible to check it redirecting output to file (>out.txt). You should use CoreConverter.StandardOutput.Read.

Let's have a form with a multiline TextBox named txtResult and a button cmdStart :

Private Sub cmdStart_Click(sender As Object, e As EventArgs) Handles cmdStart.Click
        Dim ProcInfo As New ProcessStartInfo With
            {.FileName = "C:\dbpower\dBpoweramp\CoreConverter.exe",
             .RedirectStandardOutput = True, .UseShellExecute = False,
             .WorkingDirectory = "C:\dbpower\dBpoweramp", .Arguments = "-infile=""0.mp3"" -outfile=""1.mp3"" -convert_to=""mp3 (lame)"""} '
        Dim proc As Process = Process.Start(ProcInfo)
        Dim counter As Integer = 0
        While Not proc.HasExited

            Dim a As String = ChrW(proc.StandardOutput.Read)
            If a = "*" Then
                counter += 1
                txtResult.Text = counter
                Application.DoEvents()
            End If
            Threading.Thread.Sleep(3)
            Application.DoEvents()
        End While

End Sub

The progress is written to TextBox as numberes 1-59. You can easily convert it to progress bar. Plase do not forget to change the .exe path.