Occasionally not getting output from processes running in parallel

Question

I'm trying to get the output from several process that are running in parallel. As soon as each process is done I want to print its output to the console. Unfortunatelly none of the popular approaches work.

1. Using the TaskCompletionSource

With TaskCompletionSource the output can arrive asynchronously when the Exited event is called but it's often missing because the execution does not wait for the buffers to flush. Adding a Task.Delay for a few (hundred) milliseconds doesn't seem right and also does not always work.

2. Using WaitForExit

The isssue with WaitForExit is that it indeed works in parallel but it waits for all processes to finish. Only then you can print their results so you don't see any progress until they all exited.

---

In order to demonstrate it I created a demo application. It calls a misspelled ipconfig. If you run it a few times in LINQPad you'll see at some point that this line will kick-in informing you that there was no output.

if (temp.OutputLength == 0 && temp.ErrorLength == 0) temp.Dump();

---

Here's the test application reproducing the issue. It's intentional that ipconfig is mispelled to provoke the Output problem.

void Main()
{
    var testCount = 30;

    var tasks = Enumerable.Range(0, testCount).Select(i => Task.Run(() => RunTestProcess()));

    Task.WaitAll(tasks.ToArray());

    Console.WriteLine("Done!");

}

private static object _consoleSyncLock = new object();
private static volatile int counter = 0;

public static async Task RunTestProcess()
{

    var stopwatch = Stopwatch.StartNew();
    var result = await CmdExecutor.Execute("ipconfigf", $"", "/Q", "/C");

    lock (_consoleSyncLock)
    {
        var temp = new
        {
            OutputLength = result.Output.Length,
            ErrorLength = result.Error.Length,
            Thread.CurrentThread.ManagedThreadId,
            stopwatch.Elapsed,
            Counter = counter++
        };

        if (temp.OutputLength == 0 && temp.ErrorLength == 0) temp.Dump();
    }
}

public class CmdExecutor
{
    public static Task<CmdResult> Execute(string fileName, string arguments, params string[] cmdSwitches)
    {
        Console.WriteLine(nameof(Execute) + " - " + Thread.CurrentThread.ManagedThreadId);

        if (cmdSwitches == null) throw new ArgumentNullException(nameof(cmdSwitches));
        if (fileName == null) throw new ArgumentNullException(nameof(fileName));
        if (arguments == null) throw new ArgumentNullException(nameof(arguments));

        arguments = $"{string.Join(" ", cmdSwitches)} {fileName} {arguments}";
        var startInfo = new ProcessStartInfo("cmd", arguments)
        {
            UseShellExecute = false,
            RedirectStandardOutput = true,
            RedirectStandardError = true,
            CreateNoWindow = true,
        };

        var tcs = new TaskCompletionSource<CmdResult>();

        var process = new Process { StartInfo = startInfo, EnableRaisingEvents = true };
        {
            var output = new StringBuilder();
            var error = new StringBuilder();

            process.OutputDataReceived += (sender, e) =>
            {
                output.AppendLine(e.Data);
            };

            process.ErrorDataReceived += (sender, e) =>
            {
                error.AppendLine(e.Data);
            };

            process.Exited += (sender, e) =>
            {
                tcs.SetResult(new CmdResult
                {
                    Arguments = arguments,
                    Output = output.ToString(),
                    Error = error.ToString(),
                    ExitCode = process.ExitCode
                });
                process.Dispose();
            };

            process.Start();
            process.BeginOutputReadLine();
            process.BeginErrorReadLine();

            return tcs.Task;
        }
    }
}

[Serializable]
public class CmdResult
{
    public string Arguments { get; set; }
    public string Output { get; set; }
    public string Error { get; set; }
    public int ExitCode { get; set; }
}

I don't consider this code working because it does not do what it should be doing. This is, showing the entire output of each process as soon as it's finished.

I'm curious what can be done to solve this? I don't want to simply run processes in parallel, I already can do it but I'm interested in their ouput.

Answer 1

You can wait for process to exit in Exited event itself:

process.Exited += (sender, e) =>
{
    // here
    ((Process)sender).WaitForExit();
    tcs.SetResult(new CmdResult
    {
        Arguments = arguments,
        Output = output.ToString(),
        Error = error.ToString(),
        ExitCode = process.ExitCode
    });
    process.Dispose();
};

With that addition I'm always getting entire output with your sample code.

Answer 2

To me, the error seems to be that there is the assumption that either OutputDataReceived or ErrorDataReceived will have been called at least once prior to the process.Exited - this will leave both the output and error StringBuilders empty if the process exits before the callbacks are complete.

By adding a second TaskCompletionSource to monitor the presence of at least one of the error / data callbacks, I am able to reliably always get at least one of the data callbacks invoked.

I've also changed the StringBuilders to ConcurrentBags to be safe - I'm not 100% sure of the threading used in the callbacks:

var tcsGotData = new TaskCompletionSource<bool>();
var output = new ConcurrentBag<string>();
var error = new ConcurrentBag<string>();

process.OutputDataReceived += (sender, e) =>
{
    output.Add(e.Data);
    tcsGotData.TrySetResult(true);
};

process.ErrorDataReceived += (sender, e) =>
{
    error.Add(e.Data);
    tcsGotData.TrySetResult(true);
};

process.Exited += (sender, e) =>
{
    tcsGotData.Task.Wait(); // You might want to put a timeout here, though ...
    tcs.SetResult(new CmdResult
    {
        Arguments = arguments,
        Output = string.Join("", output),
        Error = string.Join("", error),
        ExitCode = process.ExitCode
    });
    process.Dispose();
};