Are async writes to a redirected stdout synchronous or async?

Question

The documentation for Console states:

To redirect the standard input, standard output, or standard error stream, call the Console.SetIn, Console.SetOut, or Console.SetError method, respectively. I/O operations that use these streams are synchronized, which means that multiple threads can read from, or write to, the streams. This means that methods that are ordinarily asynchronous, such as TextReader.ReadLineAsync, execute synchronously if the object represents a console stream.

Does that mean that even if the output is redirected to a file, this code will be executed in a sync manner?

Console.Out.WriteLineAsync("Hello World Async");

The real world scenario is: If my program outputs logs to stdout, and it's called in a manner where the output is redirected to a file or piped to another program, will my program stall if the buffer isn't consumed in a timely manner by the OS or pipe recipient?

Alternatively, can anyone come up with a way I can determine either way?

Answer 1

If my program outputs logs to stdout, and it's called in a manner where the output is redirected to a file or piped to another program, will my program stall if the buffer isn't consumed in a timely manner by the OS or pipe recipient?

Yes, it will.

When you call Console.SetOut(), this code executes:

public static void SetOut(TextWriter newOut) {
    if (newOut == null)
        throw new ArgumentNullException("newOut");
    Contract.EndContractBlock();
#pragma warning disable 618
    new SecurityPermission(SecurityPermissionFlag.UnmanagedCode).Demand();
#pragma warning restore 618
#if FEATURE_CODEPAGES_FILE    // if no codepages file then we are locked into default codepage  and this field is not used              
    _isOutTextWriterRedirected = true;
#endif
    newOut = TextWriter.Synchronized(newOut);
    lock(InternalSyncObject) {
        _out = newOut;
    }
}

I.e., the new output TextWriter is always a writer returned by TextWriter.Syncronized(). That in turn always returns a SyncTextWriter object. And if you look at the WriteLineAsync() method for that class, it does two things:

• It is itself a synchronized method, which means if multiple threads attempt to use the writer, only one thread at a time will actually do so (i.e. all methods with that annotation are mutually exclusive between threads).
• More germane to your question though is that the WriteLineAsync() method calls the synchronous WriteLine() method on the underlying TextWriter that was used to create that SyncTextWriter object.

That second point means that the method will not return until the operation has completed. It always returns a completed task. It has no way to return an incomplete task, and if the operation is blocked for any reason, such as that the consumer of the output stream is not consuming it and thus is causing the output buffer to get filled, the call will not return.

This will cause the thread in which you make such a call to block until the write operation can complete, i.e. whatever condition was causing it to block in the first place has been resolved.

Now, all that said:

• If you are writing to a file, whether because you called SetOut() or because the process stdout has been redirected, this may slow the program down depending on how much output your program generates relative to the other work it does, but I doubt you'd experience anything characterized as a "stall". I.e. disk I/O is slow, but it doesn't generally block for extended periods of time.
• If you call SetOut() and pass some other type of writer, presumably you have control over it and can simply make sure it's written correctly and won't block when being written to.
• If the stdout is redirected externally to some other destination, such as a second .NET program running which has started your process and redirected the Process.StandardOutput stream, then it behooves that other program to ensure that it keeps up, if it cares about your program being able to continue unimpeded. But that's always the case anyway; the consumer of a process's output streams should always make sure to read them as quickly as possible.

I don't personally think it's the concern of a program to itself protect itself against redirected streams that might block, other than of course to not redirect the console streams internally to a destination object that might. Usually, this is considered the job of the receiving process, since it's the one that presumably cares whether the redirected program can do its work.

If it was really a concern, you could buffer output yourself within the process, such as having a BlockingCollection<string> object to mediate between threads that want to write to output, and a consumer thread that does the actual writing. But that's not all that useful; it just kicks the can down the road, because you don't want to keep writing to the collection indefinitely, if the consuming thread can't be counted on to be able to proceed unimpeded. If the consumer really gets blocked, the collection will just get too large and throw an OutOfMemoryException at some point. You'll have similar problems with any other mechanism that seeks to delegate the writing to a different thread, with the goal of letting the current thread run without blocking.

IMHO, better to just write normally (i.e. don't even bother using the ...Async methods, unless that's happening because you have code that abstracted the writing as the TextWriter and it's intended to work asynchronously when that's supported), and count on the consuming end of the stream to "do the right thing", and not block your process.