Netty as High Performant Http Server to Handle ~2-3 Million Requests/Sec

Question

We are trying to Solve the Problem of Handling Huge Volume of Http POST Requests, and while using Netty Server, I was able to handle only ~50K requests/sec which is too low.

My question is how to tune this Server to ensure to handle > 1.5 million requests/second?

Netty4 Server

// Configure the server.
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
        ServerBootstrap b = new ServerBootstrap();
        b.option(ChannelOption.SO_BACKLOG, 1024);

        b.group(bossGroup, workerGroup)
         .channel(NioServerSocketChannel.class)
         .handler(new LoggingHandler(LogLevel.INFO))
         .childHandler(new HttpServerInitializer(sslCtx));

        Channel ch = b.bind(PORT).sync().channel();

        System.err.println("Open your web browser and navigate to " +
                (SSL? "https" : "http") + "://127.0.0.1:" + PORT + '/');

        ch.closeFuture().sync();
    } finally {
        bossGroup.shutdownGracefully();
        workerGroup.shutdownGracefully();
    }

Initializer

    public class HttpServerInitializer extends ChannelInitializer<SocketChannel> {

    private final SslContext sslCtx;

    public HttpServerInitializer(SslContext sslCtx) {
        this.sslCtx = sslCtx;
    }

    @Override
    public void initChannel(SocketChannel ch) {
        ChannelPipeline p = ch.pipeline();
        if (sslCtx != null) {
            p.addLast(sslCtx.newHandler(ch.alloc()));
        }
        p.addLast(new HttpServerCodec());
        p.addLast("aggregator", new HttpObjectAggregator(Integer.MAX_VALUE));
        p.addLast(new HttpServerHandler());
    }
}

Handler

public class HttpServerHandler extends ChannelInboundHandlerAdapter {
private static final String CONTENT = "SUCCESS";

@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
    ctx.flush();
}

@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
    if (msg instanceof HttpRequest) {
        HttpRequest req = (HttpRequest) msg;
        final FullHttpRequest fReq = (FullHttpRequest) req;
        Charset utf8 = CharsetUtil.UTF_8;
        final ByteBuf buf = fReq.content();
        String in = buf.toString( utf8 );
        System.out.println(" In ==> "+in);
        buf.release();
        if (HttpHeaders.is100ContinueExpected(req)) {
            ctx.write(new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.CONTINUE));
        }
        in = null;
        if (HttpHeaders.is100ContinueExpected(req)) {
            ctx.write(new DefaultFullHttpResponse(HTTP_1_1, CONTINUE));
        }
        boolean keepAlive = HttpHeaders.isKeepAlive(req);
        FullHttpResponse response = new DefaultFullHttpResponse(HTTP_1_1, OK, Unpooled.wrappedBuffer(CONTENT.getBytes()));
        response.headers().set(CONTENT_TYPE, "text/plain");
        response.headers().set(CONTENT_LENGTH, response.content().readableBytes());

        if (!keepAlive) {
            ctx.write(response).addListener(ChannelFutureListener.CLOSE);
        } else {
            response.headers().set(CONNECTION, Values.KEEP_ALIVE);
            ctx.write(response);
        }
    }
}

@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
{
        cause.printStackTrace();
        ctx.close();
    }

}

Answer 1

Your question is very generic. However, I'll try to give you an answer regarding the netty optimizations and your code improvements.

Your code issues:

1. System.out.println(" In ==> "+in); - you shouldn't use this in high load cocurrent handler. Why? Because code inside println method is synchronized and thus gives penalties to your performance;
2. You do 2 class casts. To HttpRequest and to FullHttpRequest. You may use just last one;

Netty specific issues in your code:

1. You need to add epoll transport (in case your server is Linux like). It will give + ~30% out of box; How to.
2. You need to add native OpenSSL bindings. It will give + ~20%. How to.
3. EventLoopGroup bossGroup = new NioEventLoopGroup(); - you need to correctly setup sizes of bossGroup and workerGroup groups. Depending on your test scenarios. You didn't provide any info regarding your test cases, so I can't give you advice here;
4. new HttpObjectAggregator(Integer.MAX_VALUE) - you actually don't need this handler in your code. So for better performance, you may remove it.
5. new HttpServerHandler() - you don't need to create this handler for every channel. As it doesn't hold any state it may be shared across all pipelines. Search for @Sharable in netty.
6. new LoggingHandler(LogLevel.INFO) - you don't need this handler for high load tests as it logs a lot. Make your own logging when necessary;
7. buf.toString( utf8 ) - this is very wrong. You convert income bytes to string. But this doesn't make any sense as all data is already decoded in netty HttpServerCodec. So you do double work here;
8. Unpooled.wrappedBuffer(CONTENT.getBytes()) - you wrap constant message on every request. And thus - do unnecessary work on every request. You may create ByteBuf only once and do retain(), duplicate() depending on how you'll do this;
9. ctx.write(response) - you may consider using ctx.write(response, ctx.voidPromise()) in order to allocate less;

This is not all. However, fixing above issues would be a good start.