Java - Fastest Stream for TCP/HTTP Communication?

Question

I'm trying to write a Java HTTP Proxy Tunnelling program, and I need an experts advice about the best and fastest stream to use for the communication.

I've implemented the basic functionality and everything works fine. The only matter is communication speed or performance. My HTTP proxy system consists of a server program, running on a remote server and a client program running on the local machine. So far, the program looks like this:

Listener.java :

/**
 * Listens and accepts connection requests from the browser
 */
ServerSocket listener = null;
try {
    listener = new ServerSocket(port, 128);
} catch (IOException ex) {
    ex.printStackTrace(System.err);
}

ExecutorService executor = Executors.newCachedThreadPool();

Socket connection;
while (!shutdown) {
    try {
        connection = listener.accept();
        executor.execute(new ProxyTunnel(connection));
    } catch (IOException ex) {
        ex.printStackTrace(System.err);
    }
}

ProxyTunnel.java :

try {
    byte[] buffer = new byte[8192];  // 8-KB buffer
    InputStream browserInput = browser.getInputStream();
    OutputStream browserOutput = browser.getOutputStream();

    // Reading browser request ...
    StringBuilder request = new StringBuilder(2048);
    do {
        int read = browserInput.read(buffer);
        logger.log(read + " bytes read from browser.");
        if (read > 0) {
            request.append(new String(buffer, 0, read));
        }
    } while (browserInput.available() > 0 && read > 0);

    // Connecting to proxy server ...
    Socket server = new Socket(SERVER_IP, SERVER_PORT);
    server.setSoTimeout(5000);  // Setting 5 sec read timeout
    OutputStream serverOutput = server.getOutputStream();
    InputStream serverInput = server.getInputStream();

    // Sending request to server ...
    serverOutput.write(request.toString().getBytes());
    serverOutput.flush();

    // Waiting for server response ...
    StringBuilder response = new StringBuilder(16384);
    do {
        try {
            read = serverInput.read(buffer);
        } catch (SocketTimeoutException ex) {
            break; // Timeout!
        }
        if (read > 0) {
            // Send response to browser.");
            response.append(new String(buffer, 0, read));
            browserOutput.write(buffer, 0, read);
            browserOutput.flush();
        }
    } while (read > 0);

    // Closing connections ...
    server.close();

} catch (IOException ex) {
    ex.printStackTrace(System.err);
} finally {
    try {
        browser.close();
    } catch (IOException ex) {
        ex.printStackTrace(System.err);
    }
}

The server program uses a similar fashion and sends the HTTP request to the destination server (e.g. www.stackoverflow.com) and forwards the response to the client program, where the client program forwards the response to the local browser.

1. How can I improve the performance of these TCP/HTTP communications?
2. Does using buffered streams such as BufferedInputSream and BufferedOutputStream improve communication performance?
3. Will I gain any performance improvements if I use java.nio Channels and Buffers, instead of java.net Sockets and java.io Stream?

Answer 1

Don't do it yourself

Advice 0: there are plenty of proxy servers out there, much more scalable, stable and mature. Do you really need to write your own?

Don't use StringBuilder/String to buffer request

byte[] buffer = new byte[8192];  // 8-KB buffer
//...
browserInput.read(buffer);
//...
request.append(new String(buffer, 0, read));
//...
serverOutput.write(request.toString().getBytes());

This is flawed for several reasons:

• you are assuming your HTTP calls are text (ASCII) only, binary data will be malformed after transforming to String and back to byte[], see: String, byte[] and compression

• even if the protocol is text-based, you are using system's default encoding. I bet this is not what you want

• finally, the most important part: do not buffer the whole request. Read chunk of data from incoming request and forward it immediately to target server in one iteration. There is absolutely no need for the extra memory overhead and latency. Immediately after receiving few bytes dispatch them and forget about them.

Don't use Executors.newCachedThreadPool()

This pool can grow inifinitely, creating thousands of threads during peak. Essentially you create one thread per connection (except that the pool reuses free threads, but creates new if none available). Consider Executors.newFixedThreadPool(100) - 100-200 threads should be enough in most cases. Above that you'll most likely burn your CPU barely in context switching, without doing much work. Don't be afraid of latency, scale out.

Use non-blocking netty stack

Which brings us to the final advice. Drop blocking sockets altogether. They are handy, but don't scale well due to thread-per-connection requirement. Too much memory is spent to hold stack, too much CPU is wasted for context switching. netty is great and it builds powerful abstraction over NIO.

Check out the examples, they include HTTP client/server code. There is a bit of a learning curve, but you can expect performance growth by several order of magnitued.