Tweaking performance of Java's sockets

Question

I've created a remote desktop control application. Obviously, it consists of client and server parts:

Server:

• Receiving mouse/keyboard actions from client;
• Sending screenshot of desktop to client.

Client:

• Receiving screenshot from server;
• Sending mouse/keyboard actions;

Consider sending the screenshot. When I use my home PC as a server - I end up with a 1920x1080 screenshot dimension. By using JAI Image I/O Tools and encoding it as PNG I was able to achieve the following stats for such a big image:

1. Write time ~0.2 s; (not into socket, but into some "regular" output stream, i. e. encode time)
2. Read time ~0.05 s; (not from socket, but from some "regular" input stream, i. e. decode time)
3. Size ~250 KB;
4. Perfect quality.

As a result, according to #1 - the ideal possible FPS should be ~5.

Unfortunately, I'm not able to achieve even those ~5 FPS, not even 2 FPS. I searched for the bottleneck and found out that writing/reading to/from socket I/O streams takes up to ~2 s (See appendix 1 and 2 for clarification). Surely that is unacceptable.

I've researched the topic a bit - and added buffering of socket's I/O streams (with BufferedInputStream and BufferedOutputStream) on both sides. I've started with 64 KB sizes. This indeed improved performance. But still can't have at least 2 FPS! In addition, I've experimented with Socket#setReceiveBufferSize and Socket#setSendBufferSize and there were some changes in speed, but I don't know how exactly they behave, therefore I don't know which values to use.

Look at initialization code:

Server:

    ServerSocket serverSocket = new ServerSocket();
    serverSocket.setReceiveBufferSize( ? ); // #1
    serverSocket.bind(new InetSocketAddress(...));

    Socket clientSocket = serverSocket.accept();
    clientSocket.setSendBufferSize( ? ); // #2
    clientSocket.setReceiveBufferSize( ? ); // #3

    OutputStream outputStream = new BufferedOutputStream(
            clientSocket.getOutputStream(), ? ); // #4
    InputStream inputStream = new BufferedInputStream(
            clientSocket.getInputStream(), ? ); // #5

Client:

    Socket socket = new Socket(...);
    socket.setSendBufferSize( ? ); // #6
    socket.setReceiveBufferSize( ? ); // #7

    OutputStream outputStream = new BufferedOutputStream(
            socket.getOutputStream(), ? ); // #8
    InputStream inputStream = new BufferedInputStream(
            socket.getInputStream(), ? ); // #9

Questions:

1. Which values would you recommend (to improve performance) for all these cases and why?
2. Please clarify Socket#setReceiveBufferSize and Socket#setSendBufferSize behavior.
3. What other methods/techniques can you advise to improve performance of such application?
4. Skype provides good-quality real-time desktop transmission - how do they do it?

Appendix 1: Adding the unrolled pseudo-code of client socket reading (@mcfinnigan):

while(true) {
    // objectInputStream is wrapping socket's buffered input stream.
    Object object = objectInputStream.readObject(); // <--- Bottleneck (without setting proper buffer sizes is ~2 s)

    if(object == null)
        continue;

    if(object.getClass() == ImageCapsule.class) {
        ImageCapsule imageCapsule = (ImageCapsule)object;

        screen = imageCapsule.read(); // <--- Decode PNG (~0.05 s)

        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                repaint();
            }
        });
    }
}

Appendix 2: Adding the unrolled pseudo-code of server socket writing (@EJP):

while(true) {
    // objectOutputStream is wrapping socket's buffered output stream.
    BufferedImage screen = ... // obtaining screenshot
    ImageCapsule imageCapsule = new ImageCapsule();

    imageCapsule.write(screen, formatName()); // <--- Encode PNG (~0.2 s)

    try {
        objectOutputStream.writeObject(imageCapsule); // <--- Bottleneck (without setting proper buffer sizes is ~2 s)
    }
    finally {
        objectOutputStream.flush();
        objectOutputStream.reset(); // Reset to free written objects.
    }
}

Conclusion:

Thanks for your answers, specially EJP - he made things a bit more clear to me. If you are like me - seeking answers on how to tweak sockets' performance you should definitely check TCP/IP Sockets in Java, Second Edition: Practical Guide for Programmers, especially chapter 6 "Under the Hood" which describes what happens behind the scenes of *Socket classes, how send and receive buffers are managed and utilized (which are primary keys to performance).

Answer 1

• Write time ~0.2 s;
• Read time ~0.05 s;

These objectives are completely meaningless without consideration of the latency and bandwidths of the intervening network.

Size ~250 KB;

Off topic. Image size is up to you, and it has no relationship to actual programming, which is the purpose of this site.

Perfect quality.

'Perfect quality' only requires that you don't drop any bits, which you won't get anyway via TCP.

  serverSocket.setReceiveBufferSize( ? ); // #1

That sets the receive buffer size for all accepted sockets. Set it as large as you can afford, over 64k if possible.

socket.setSendBufferSize( ? ); // #6

Set this as large as you can afford, over 64k if possible.

    socket.setReceiveBufferSize( ? ); // #7

As this is an accepted socket you have already done this above. Remove.

    OutputStream outputStream = new BufferedOutputStream(
            socket.getOutputStream(), ? ); // #8
    InputStream inputStream = new BufferedInputStream(
            socket.getInputStream(), ? ); // #9

The defaults for these are 8k; as long as you have decent socket buffer sizes that should be enough.

Which values would you recommend (to improve performance) for all these cases and why?

See above.

Please clarify Socket#setReceiveBufferSize() and Socket#setSendBufferSize() behavior.

They control the size of the TCP 'window'. It's a fairly abstruse topic but the idea is to get the size at least equal to the bandwidth-delay product of your network, i.e. bandwidth in bytes/second times delay in seconds >= buffer size in bytes.

What other methods/techniques can you advise to improve performance of such application?

Don't futz around with sleeps and doing other tasks while sending data. Send it as fast as you can in the tightest possible loop you can arrange.

Skype provides good-quality real-time desktop transmission - how do they do it?

Off topic and probably unknowable unless a Skype employee happens to want to give away company secrets here.

Answer 2

• If client opens new TCP connection for each image, then TCP slow start may cause the slowness. --> Use same TCP connection for all frames.

• TCP have own buffers which are used optimal way for TCP --> BufferedOutputStream gives sometimes benefit and some times not.

• Usually only small part of screen is changed between screenshots. --> Transfer only changed parts.