Console.WriteLine slow

Question

I run through millions of records and sometimes I have to debug using Console.WriteLine to see what is going on.

However, Console.WriteLine is very slow, considerably slower than writing to a file.

BUT it is very convenient - does anyone know of a way to speed it up?

Answer 1

If it is just for debugging purposes you should use Debug.WriteLine instead. This will most likely be a bit faster than using Console.WriteLine.

Example

Debug.WriteLine("There was an error processing the data.");

Answer 2

You can use the OutputDebugString API function to send a string to the debugger. It doesn't wait for anything to redraw and this is probably the fastest thing you can get without digging into the low-level stuff too much. The text you give to this function will go into Visual Studio Output window.

[DllImport("kernel32.dll")]
static extern void OutputDebugString(string lpOutputString);

Then you just call OutputDebugString("Hello world!");

Answer 3

Do something like this:

public static class QueuedConsole
{
    private static StringBuilder _sb = new StringBuilder();
    private static int _lineCount;

    public void WriteLine(string message)
    {
        _sb.AppendLine(message);
        ++_lineCount;
        if (_lineCount >= 10)
           WriteAll();
    }

    public void WriteAll()
    {
        Console.WriteLine(_sb.ToString());
        _lineCount = 0;
        _sb.Clear();
    }
}

QueuedConsole.WriteLine("This message will not be written directly, but with nine other entries to increase performance.");

//after your operations, end with write all to get the last lines.
QueuedConsole.WriteAll();

Here is another example: Does Console.WriteLine block?

Answer 4

I recently did a benchmark battery for this on .NET 4.8. The tests included many of the proposals mentioned on this page, including Async and blocking variants of both BCL and custom code, and then most of those both with and without dedicated threading, and finally scaled across power-of-2 buffer sizes.

The fastest method, now used in my own projects, buffers 64K of wide (Unicode) characters at a time from .NET directly to the Win32 function WriteConsoleW without copying or even hard-pinning. Remainders larger than 64K, after filling and flushing one buffer, are also sent directly, and in-situ as well. The approach deliberately bypasses the Stream/TextWriter paradigm so it can (obviously enough) provide .NET text that is already Unicode to a (native) Unicode API without all the superfluous memory copying/shuffling and byte[] array allocations required for first "decoding" to a byte stream.

If there is interest (perhaps because the buffering logic is slightly intricate), I can provide the source for the above; it's only about 80 lines. However, my tests determined that there's a simpler way to get nearly the same performance, and since it doesn't require any Win32 calls, I'll show this latter technique instead.

The following is way faster than Console.Write:

public static class FastConsole
{
    static readonly BufferedStream str;

    static FastConsole()
    {
        Console.OutputEncoding = Encoding.Unicode;  // crucial

        // avoid special "ShadowBuffer" for hard-coded size 0x14000 in 'BufferedStream' 
        str = new BufferedStream(Console.OpenStandardOutput(), 0x15000);
    }

    public static void WriteLine(String s) => Write(s + "\r\n");

    public static void Write(String s)
    {
        // avoid endless 'GetByteCount' dithering in 'Encoding.Unicode.GetBytes(s)'
        var rgb = new byte[s.Length << 1];
        Encoding.Unicode.GetBytes(s, 0, s.Length, rgb, 0);

        lock (str)   // (optional, can omit if appropriate)
            str.Write(rgb, 0, rgb.Length);
    }

    public static void Flush() { lock (str) str.Flush(); }
};

Note that this is a buffered writer, so you must call Flush() when you have no more text to write.

I should also mention that, as shown, technically this code assumes 16-bit Unicode (UCS-2, as opposed to UTF-16) and thus won't properly handle 4-byte escape surrogates for characters beyond the Basic Multilingual Plane. The point hardly seems important given the more extreme limitations on console text display in general, but could perhaps still matter for piping/redirection.

Usage:

FastConsole.WriteLine("hello world.");
// etc...
FastConsole.Flush();

On my machine, this gets about 77,000 lines/second (mixed-length) versus only 5,200 lines/sec under identical conditions for normal Console.WriteLine. That's a factor of almost 15x speedup.

These are controlled comparison results only; note that absolute measurements of console output performance are highly variable, depending on the console window settings and runtime conditions, including size, layout, fonts, DWM clipping, etc.

Answer 5

A little old thread and maybe not exactly what the OP is looking for, but I ran into the same question recently, when processing audio data in real time.

I compared Console.WriteLine to Debug.WriteLine with this code and used DebugView as a dos box alternative. It's only an executable (nothing to install) and can be customized in very neat ways (filters & colors!). It has no problems with tens of thousands of lines and manages the memory quite well (I could not find any kind of leak, even after days of logging).

After doing some testing in different environments (e.g.: virtual machine, IDE, background processes running, etc) I made the following observations:

• Debug is almost always faster
• For small bursts of lines (<1000), it's about 10 times faster
• For larger chunks it seems to converge to about 3x
• If the Debug output goes to the IDE, Console is faster :-)
• If DebugView is not running, Debug gets even faster
• For really large amounts of consecutive outputs (>10000), Debug gets slower and Console stays constant. I presume this is due to the memory, Debug has to allocate and Console does not.
• Obviously, it makes a difference if DebugView is actually "in-view" or not, as the many gui updates have a significant impact on the overall performance of the system, while Console simply hangs, if visible or not. But it's hard to put numbers on that one...

I did not try multiple threads writing to the Console, as I think this should generally avoided. I never had (performance) problems when writing to Debug from multiple threads.

If you compile with Release settings, usually all Debug statements are omitted and Trace should produce the same behaviour as Debug.

I used VS2017 & .Net 4.6.1

Sorry for so much code, but I had to tweak it quite a lot to actually measure what I wanted to. If you can spot any problems with the code (biases, etc.), please comment. I would love to get more precise data for real life systems.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;

namespace Console_vs_Debug {
 class Program {
  class Trial {
   public string name;
   public Action console;
   public Action debug;
   public List < float > consoleMeasuredTimes = new List < float > ();
   public List < float > debugMeasuredTimes = new List < float > ();
  }

  static Stopwatch sw = new Stopwatch();
  private static int repeatLoop = 1000;
  private static int iterations = 2;
  private static int dummy = 0;

  static void Main(string[] args) {
   if (args.Length == 2) {
    repeatLoop = int.Parse(args[0]);
    iterations = int.Parse(args[1]);
   }

   // do some dummy work
   for (int i = 0; i < 100; i++) {
    Console.WriteLine("-");
    Debug.WriteLine("-");
   }

   for (int i = 0; i < iterations; i++) {
    foreach(Trial trial in trials) {
     Thread.Sleep(50);
     sw.Restart();
     for (int r = 0; r < repeatLoop; r++)
      trial.console();
     sw.Stop();
     trial.consoleMeasuredTimes.Add(sw.ElapsedMilliseconds);
     Thread.Sleep(1);
     sw.Restart();
     for (int r = 0; r < repeatLoop; r++)
      trial.debug();
     sw.Stop();
     trial.debugMeasuredTimes.Add(sw.ElapsedMilliseconds);

    }
   }
   Console.WriteLine("---\r\n");
   foreach(Trial trial in trials) {
    var consoleAverage = trial.consoleMeasuredTimes.Average();
    var debugAverage = trial.debugMeasuredTimes.Average();
    Console.WriteLine(trial.name);
    Console.WriteLine($ "    console: {consoleAverage,11:F4}");
    Console.WriteLine($ "      debug: {debugAverage,11:F4}");
    Console.WriteLine($ "{consoleAverage / debugAverage,32:F2} (console/debug)");
    Console.WriteLine();
   }

   Console.WriteLine("all measurements are in milliseconds");
   Console.WriteLine("anykey");
   Console.ReadKey();
  }

  private static List < Trial > trials = new List < Trial > {
   new Trial {
    name = "constant",
     console = delegate {
      Console.WriteLine("A static and constant string");
     },
     debug = delegate {
      Debug.WriteLine("A static and constant string");
     }
   },
   new Trial {
    name = "dynamic",
     console = delegate {
      Console.WriteLine("A dynamically built string (number " + dummy++ + ")");
     },
     debug = delegate {
      Debug.WriteLine("A dynamically built string (number " + dummy++ + ")");
     }
   },
   new Trial {
    name = "interpolated",
     console = delegate {
      Console.WriteLine($ "An interpolated string (number {dummy++,6})");
     },
     debug = delegate {
      Debug.WriteLine($ "An interpolated string (number {dummy++,6})");
     }
   }
  };
 }
}

Answer 6

Why Console is slow:

• Console output is actually an IO stream that's managed by your operating system. Most IO classes (like FileStream) have async methods but the Console class was never updated so it always blocks the thread when writing.

• Console.WriteLine is backed by SyncTextWriter which uses a global lock to prevent multiple threads from writing partial lines. This is a major bottleneck that forces all threads to wait for each other to finish the write.

• If the console window is visible on screen then there can be significant slowdown because the window needs to be redrawn before the console output is considered flushed.

Solutions:

Wrap the Console stream with a StreamWriter and then use async methods:

var sw = new StreamWriter(Console.OpenStandardOutput());
await sw.WriteLineAsync("...");

You can also set a larger buffer if you need to use sync methods. The call will occasionally block when the buffer gets full and is flushed to the stream.

// set a buffer size
var sw = new StreamWriter(Console.OpenStandardOutput(), Encoding.UTF8, 8192);
// this write call will block when buffer is full
sw.Write("...")

If you want the fastest writes though, you'll need to make your own buffer class that writes to memory and flushes to the console asynchronously in the background using a single thread without locking. The new Channel<T> class in .NET Core 2.1 makes this simple and fast. Plenty of other questions showing that code but comment if you need tips.

Answer 7

Just a little trick I use sometimes: If you remove focus from the Console window by opening another window over it, and leave it until it completes, it won't redraw the window until you refocus, speeding it up significantly. Just make sure you have the buffer set up high enough that you can scroll back through all of the output.

Answer 8

Here is a 7 times faster implementation that bulk-writes to the Console, with a 10 msec delay. The downside is that you must remember to call Console2.Flush() at the end of the program, otherwise you may lose some output.

public static class Console2
{
    private static readonly StringBuilder _sb = new StringBuilder();
    private static volatile CancellationTokenSource _cts;
    private static int _count;

    public static void Write(string value)
    {
        lock (_sb) _sb.Append(value);
        ScheduleFlush();
    }
    public static void Write(string format, params object[] args)
    {
        lock (_sb) _sb.AppendFormat(format, args);
        ScheduleFlush();
    }
    public static void WriteLine(string value)
        => Write(value + Environment.NewLine);

    public static void WriteLine(string format, params object[] args)
        => Write(format + Environment.NewLine, args);

    public static void WriteLine()
        => WriteLine("");

    private static void ScheduleFlush()
    {
        _cts?.Cancel();
        var count = Interlocked.Increment(ref _count);
        if (count % 100 == 0) // periodically flush without cancellation
        {
            var fireAndForget = Task.Run(Flush);
        }
        else
        {
            _cts = new CancellationTokenSource();
            var token = _cts.Token;
            var fireAndForget = Task.Run(async () =>
            {
                await Task.Delay(10, token);
                Flush();
            }, token);
        }
    }

    public static void Flush()
    {
        _cts?.Cancel();
        string text;
        lock (_sb)
        {
            if (_sb.Length == 0) return;
            text = _sb.ToString();
            _sb.Clear();
        }
        Console.Write(text);
    }
}

Usage example:

for (int i = 1; i <= 1000; i++)
{
    Console2.WriteLine($"{DateTime.Now:HH:mm:ss.fff} > Line {i}");
}
Console2.Flush();

Output:

06:27:22.882 > Line 1
06:27:22.882 > Line 2
...
06:27:22.893 > Line 999
06:27:22.893 > Line 1000

Answer 9

Try using the System.Diagnostics Debug class? You can accomplish the same things as using Console.WriteLine.

You can view the available class methods here.