Java: socket messages lost

Question

We're developing a server application in Java (1.6) which is a transactional server, that listen for connections through TCP sockets. Each new connections create a new thread that keeps alive until the connection is closed. Each client will send transactions to the server that will be processed and then the response is sent back to the client.

This works fine. Problem now arises when we want to send many async transactions (or messages) through the same socket. I've written a small application that send 1000 transactions with an interval of 10 ms between each transaction. The application is async, so messages are sent and the responses comes in the middle.

This is the code from the part that handles the incomming messages and send them to another component to be processed (this component has a thread pool):

public void run() {
...
...

  socketBuf = new BufferedInputStream(input);
  baos = new ByteArrayOutputStream();

  while ((bytes_read = socketBuf.read(buffer)) != -1) {

    if (bytes_read < 0) {
        log.error("Tried to read from socket, read() returned < 0,  Closing socket.");
        return;
    }

    baos.write(buffer, 0, bytes_read);
    break;
  }

  if (bytes_read >= 0) {

    baos.flush();
    byte data[] = baos.toByteArray();

    if (data.length > 0) {                      
        GWTranData tData = posMessage.decode(data, false);   
        if (tData.getMessageType() > 0) {

            // Send to the Pre-Online Manager to be processed                       
            PreOnlineJob newJob = new PreOnlineJob(tData);
            newJob.addJobStatusListener(this);
            GWServer.getPreOnlineInstance().addJob(newJob);
        }
    }
  }
  else {
    clientSocket.close();
    break;
  }

} while(true);
  } 

The problem we're facing when sending many transactions in a brief time is that some messages are lost and don't reach the server. Doing a deep analysis, we've found that when messages are sent too fast, there're more than one message in the buffer, so data[] has two or more messages but only one will be performed. The size of the message sent is 200 bytes, so a buffer of 512 is more than enough.

Is there any problem with the way I've implemented the socket read? Is there a better way?

Thanks guys.

Answer 1

The problem is with the way you consume bytes read from the socket. Your assumption is that you get one "message" per read. That assumption is wrong - TCP does not know about your application message boundaries, but gives you a stream of bytes, so you can get several messages at once, or a part of a message, or both.

You have to buffer unprocessed part of the received stream, check if you got complete message, read some more until you do, process the message, and continue in a loop.

Edit 0:

There are several ways to design your application-level protocol on top of TCP:

• fixed length messages (easy),
• delimited messages (needs explicit byte-sequence to designate end/start of message, like SOH in FIX, or \r\n in HTTP),
• length-prefixed messages as @Thomas suggests in comments,
• "self-describing" messages - like s-expressions or what not, need to be parsed,
• maybe others.