Very Strange Problem sending data via Sockets in C#

Question

I appologize for this lengthy post. I have made it as small as possible while still conveying the problem.

Ok this is driving me crazy. I have a client and a server program, both in C#. The server sends data to the client via Socket.Send(). The client receives the data via Socket.BeginReceive and Socket.Receive. My pseudo-protocol is as follows: The server sends a two-bye (short) value indicating the length of the actual data followed immediately by the actual data. The client reads the first two byes asynchronously, converts the bytes to a short, and immediately reads that many bytes from the socket synchronously.

Now this works fine for one cycle every few seconds or so but when I increase the speed, things get weird. It seems that the client will randomly read the actual data when it is attempting to read from the two-byte length. It then tries to convert these arbitrary two bytes into a short which leads to a completely incorrect value, causing a crash. The following code is from my program but trimmed to only show the important lines.

Server-side method for sending data:

private static object myLock = new object();
private static bool sendData(Socket sock, String prefix, byte[] data)
{
    lock(myLock){
        try
        {
            // prefix is always a 4-bytes string
            // encoder is an ASCIIEncoding object    
            byte[] prefixBytes = encoder.GetBytes(prefix);
            short length = (short)(prefixBytes.Length + data.Length);

            sock.Send(BitConverter.GetBytes(length));
            sock.Send(prefixBytes);
            sock.Send(data);

            return true;
        } 
        catch(Exception e){/*blah blah blah*/}
    }
}

Client-side method for receiving data:

private static object myLock = new object();
private void receiveData(IAsyncResult result)
{
    lock(myLock){
        byte[] buffer = new byte[1024];
        Socket sock = result.AsyncState as Socket;
        try
        {
            sock.EndReceive(result);
            short n = BitConverter.ToInt16(smallBuffer, 0);
            // smallBuffer is a 2-byte array

            // Receive n bytes
            sock.Receive(buffer, n, SocketFlags.None);

            // Determine the prefix.  encoder is an ASCIIEncoding object
            String prefix = encoder.GetString(buffer, 0, 4);

            // Code to process the data goes here

            sock.BeginReceive(smallBuffer, 0, 2, SocketFlags.None, receiveData, sock);
        }
        catch(Exception e){/*blah blah blah*/}
    }
}

Server-side code to recreate the issue reliably:

byte[] b = new byte[1020];  // arbitrary length

for (int i = 0; i < b.Length; i++)
    b[i] = 7;  // arbitrary value of 7

while (true)
{
    sendData(socket, "PRFX", b);
    // socket is a Socket connected to a client running the same code as above
    // "PRFX" is an arbitrary 4-character string that will be sent
}

Looking at the code above, one can determine that the server will forever send the number 1024, the length of the total data, including the prefix, as a short( 0x400) followed by "PRFX" in ASCII binary followed by a bunch of 7's (0x07). The client will forever read the first two bytes (0x400), interpret that as being 1024, store that value as n, and then read 1024 byes from the stream.

This is indeed what it does for the first 40 or so iterations but, spontaneously, the client will read the first two byes and interpret them as being 1799, not 1024! 1799 in hex is 0x0707 which is two consecutive 7's!!! That is the data, not the length! What happened to those two bytes? This happens with whatever value I put in the byte array, I just chose 7 because it is easy to see the correlation with 1799.

If you are still reading by this point, I applaud your dedication.

Some important observations:

• Decreasing the length of b will increase the number of iterations before the problem occurs but does not prevent the problem from occurring
• Adding a significant delay between each iteration of the loop may prevent the problem from occurring
• This DOES NOT happen when using both the client and server on the same host and connecting via a loopback address.

As mentioned, this is driving me crazy! I am always able to solve my programming problems but this one has completely stumped me. So I am here pleading for any advise or knowledge on this subject.

Thank you.

Answer 1

I suspect you're assuming that the entire message is delivered in one call to receiveData. This is not, in general, the case. Fragmentation, delays, etc can end up meaning that the data arrives in dribbles, so that you might get your receiveData function called when there's only, e.g. 900 bytes ready. Your call sock.Receive(buffer, n, SocketFlags.None); says "Give me the data into the buffer, up to n bytes" - you might receive fewer, and the number of bytes actually read is returned by Receive.

This explains why decreasing b, adding more delay or using the same host seems to "fix" the problem - the odds are vastly increased that the entire message will arrive in one go using these methods (smaller b, smaller data; adding a delay means there's less overall data in the pipe; there's no network in the way for the local address).

To see if this is indeed the problem, log the return value of Receive. It will occasionally be less than 1024 if my diagnosis is correct. To fix it, you need to either wait until the local network stack's buffer has all your data (not ideal), or just receive data as and when it arrives, storing it locally until a whole message is ready and processing it then.

Answer 2

It sounds to me like the main problem here is not checking the result of EndReceive which is the number of bytes read. If the socket is open, this can be anything > 0 and <= the max you specified. It is not safe to assume that because you asked for 2 bytes, 2 bytes were read. Ditto when reading the actual data.

You must always loop, accumulating the amount of data you expect (and decreasing the remaining count, and increasing the offset, accordingly).

The mix of sync/async won't help make it easy, either :£