Java socket performance bottleneck: where?

Question

I recently started the development of an application making intensive usage of networking. A first attempt was made using RMI and for a couple of reasons, we switched over to pure sockets. However, when testing sockets over a network, or even on localhost, we dropped to a rate of 25 requests/second. While using RMI it was two orders of magnitude higher.

With a little more testing, we obtained following (for localhost):

• Sending always the same object: 31628 requests/sec
• Sending always a new object: 25 requests/sec
• Only object creation rate: 3-4 millions per second (so that is not the bottleneck)

Here is the client code: (the server side just replies an "ACK")

public static void main(String[] args) throws IOException, ClassNotFoundException
{
    Socket kkSocket = null;
    ObjectOutputStream out = null;
    ObjectInputStream in = null;


    kkSocket = new Socket("barium", 4444);
    out = new ObjectOutputStream(kkSocket.getOutputStream());
    in = new ObjectInputStream(kkSocket.getInputStream());


    long throughput;
    long millis;

    TcpRequest hello = null;


    throughput = 0;
    millis = System.currentTimeMillis();
    while (System.currentTimeMillis() < millis + 1000)
    {
        hello = new TcpRequest();
        hello.service = "hello";
        hello.payload = Math.random();
        throughput++;
    }

    System.out.println("-------------------------------------------------------");
    System.out.println("|        Objects created: " + (throughput)  + " requests/sec.");
    System.out.println("-------------------------------------------------------");


    throughput = 0;
    millis = System.currentTimeMillis();
    while (System.currentTimeMillis() < millis + 1000)
    {
        out.writeObject(hello);
        Object res = in.readObject();
        throughput++;
    }
    System.out.println("-------------------------------------------------------");
    System.out.println("|        Same object throughput: " + (throughput)  + " requests/sec.");
    System.out.println("-------------------------------------------------------");


    throughput = 0;
    millis = System.currentTimeMillis();
    while (System.currentTimeMillis() < millis + 1000)
    {
        hello = new TcpRequest();
        out.writeObject(hello);
        Object res = in.readObject();
        throughput++;
    }
    System.out.println("-------------------------------------------------------");
    System.out.println("|        New objetcs throughput: " + (throughput)  + " requests/sec.");
    System.out.println("-------------------------------------------------------");


    out.close();
    in.close();

    kkSocket.close();
}

The TcpRequest class is just a dummy class without anything special.

So, if creating object is fast, if sending it over the network is fast ...why on earth is sending a new object over the network so slow?!?!

And if you keep a same object and modify its content before sending it, you will also have high transfer rate ...but fall in the nasty pitfall:

When working with object serialization it is important to keep in mind that the ObjectOutputStream maintains a hashtable mapping the objects written into the stream to a handle. When an object is written to the stream for the first time, its contents will be copied to the stream. Subsequent writes, however, result in a handle to the object being written to the stream.

...which happened to us and caused some hours of debugging before figuring it out.

So basically ...how do you achieve high throughput with sockets? (...I mean, with RMI being a wrapper around it we were already two orders of magnitude higher!)

SOLVED:

By replacing:

out = new ObjectOutputStream(kkSocket.getOutputStream());

With:

out = new ObjectOutputStream(new BufferedOutputStream(kkSocket.getOutputStream()))

The performances are normal again (nearly the same high throughput as with the same object case)

Answer 1

Found it:

Instead of:

out = new ObjectOutputStream(kkSocket.getOutputStream());

You should use:

out = new ObjectOutputStream(new BufferedOutputStream(kkSocket.getOutputStream()));

And

out.flush();

when sending a message.

...makes a huge difference ...though I don't know exactly why.