C# BinaryReader.ReadChar throws "System.ArgumentException: The output char buffer is too small" when reading NetworkStream

Question

When reading C# NetworkStream (from stream-type TCP socket), BinaryReader.ReadChar occasionally throws exceptions System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode (UTF-8)'

All buffers have their default sizes (none of them are manually set) and setting bigger buffer sizes does not affect the issue.

And what is completely discouraging:

• The exception does not occur when using breakpoint and walking step-by-step through the line with ReadChar call

• The exception does not occur if ReadChar is preceded by Thread.Sleep(1000) (but can still occur with smaller timeouts)

• The exception does not occur when using BinaryReader on FileStream, where all precise bytes of TCP-server's answer are stored.

So, what can be the time-related issue with buffering single characters from stream a socket?

Answer 1

I had this issue too. And here are some facts about it:

1. System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode (UTF-8)' is known to be related to UTF-8 encoding problem (invalid character code) rather than to buffering problem - Detials here

2. NetworkStream (Read and other methods) is known to return only the amount of bytes which is already present in system network buffers instead of blocking until all requested data will be recieved - Details here. So, one needs to use Read in a loop to get all requested data

3. BinaryReader is known to throw an exception when getting less data from NetworkStream than it expected, instead of using a loop to retrieve the rest (and YES, I am sure, this means a bug!) - Details here

So, my solution was to partially reimplement BinaryReader (I have called my class BinReader) adding some useful features and making proper Read method with a loop:

public int Read( byte[] buf, int off, int count ) {
    int read = 0;
    while( read < count ) {
        int toread = count - read;
        int portion = BaseStream.Read( buf, off, toread );
        read += portion;
        off += portion;
    }
    return read;
}

That has solved it for me.