Fast reading of console input

Question

I need for fast reading data from standard input stream of console. Input consist of 100.000 rows with 20 chars each (2 million chars); user paste it from clipboard. My procedure works for about 3 minutes (very slowly; the target is 10 seconds). It is look like:

var inputData = new string[100000]; // 100.000 rows with 20 chars
for (int i = 0; i < 100000; i++) // Cycle duration is about 3 minutes...
{
    inputData[i] = Console.ReadLine();
}
// some processing...

What's I tried:

1. Directly: Console.Read, Console.ReadKey - the same result

2. Console.In: Read(), ReadLine(), ReadAsync(), ReadLineAsync(), ReadBlock(with various block size), ReadBlockAsync(), ReadToEnd(), ReadToEndAsync() - the same result

3. new StreamReader(Console.OpenStandardInput(buffer)) with various buffer and block size - the same result

4. Hide console window at start of reading, and show it when reading is finished - acceleration 10%

5. I tried get input data from file - it's works perfectly and fast. But I need read from __ConsoleStream.

I noticed, while input reading in progress - process conhost.exe actively uses a processor.

How can I speed up the reading of input?

upd:

6. Increasing/decreasing Console.BufferHeight and Console.BufferWidth has no effect

7. ReadFile msdn is also slowly. But I noticed an interesting fact:

   ReadFile(handle, buffer, bufferSize, out bytesCount, null);
   // bufferSize may be very big, but buffer obtains no more than one row (with \r\n).
   // So, it seems that data passed into InputStream row-by-row syncroniously.
   

Answer 1

In you scenario a lot of time is wasted by attempts to display inserting symbols. You can disable inserting symbols displaying in Windows (I don't know how to do that on other platforms).

Unfortunately, necessary API is not exposed by .NET (at least in 4.6.1). So you need following native methods/constants:

internal class NativeMethods
{
    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern bool SetConsoleMode(IntPtr hConsoleHandle, int mode);

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern bool GetConsoleMode(IntPtr hConsoleHandle, out int mode);

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern IntPtr GetStdHandle(int nStdHandle);

    internal const int STD_INPUT_HANDLE = -10;
    internal const int ENABLE_ECHO_INPUT = 0x0004;
}

and use them in following way before receiving data from clipboard:

var handle = NativeMethods.GetStdHandle(NativeMethods.STD_INPUT_HANDLE);
int mode; 
NativeMethods.GetConsoleMode(handle, out mode);
mode &= ~NativeMethods.ENABLE_ECHO_INPUT; // disable flag
NativeMethods.SetConsoleMode(handle, mode);

Don't forget to revert console mode flags back when you finished receiving clipboard data. I hope it will reduce your performance problem. More info about console modes can be found on GetConsoleMode

Further attempts to optimize can include:

• Rewrite console reading code without locks (as it implemented in .NET) and ensure that no any threads works with console at that moment. Quite expensive task.
• Try to find a way to increase stdin buffer size. But i'm not sure is it possible at all.
• Don't forget to test in release build without debugging %)