Netty's ability to handle large payloads

Question

I am using Apache MINA in my open source project. However, I have hit a performance road block where it is not being able to handle large payloads uploaded to the server from mobile devices. I am getting Out of Memory errors. I have never used Netty and looking to swap MINA with Netty.

My question for the Netty team is that how does Netty perform under a load of large sized incoming payloads. In my performance tests, I am generating the load using a 1000 concurrent devices sending 1MByte sized payloads each. It tends to generate 1GIG of uploaded data in 5 minutes. This load tends to break MINA.

Can Netty support this load and more. Your input on this is greatly appreciated.

btw: I did profile the application using JProfiler. It does point to the MINA stack as the culprit. I even completely disconnected my server side stack and left it at plain MINA server that consumes data and returns data. In this mode, even though there is no real processing going on with my backend, it starts to run out of memory in about 15 to 20 minutes

Answer 1

It would seem to be as an incorrect usage of the library rather than an actual problem with the library itself. If not user error, IMO Netty would probably do the same thing. I've used MINA in high bandwidth applications before without this result. It may be that your network is saturated, and the buffers just consume all of the memory waiting to upload. Just my 2¢.