How to optimize Java gc for long living objects

Question

My java app maintains an internal cache that could grow up to 10 gigs. Expiration policy is set to 30 minutes or when memory threshold is reached (I'm using local ehcache). It is obvious that after 30 minutes all cached object will be in the old gen and it will require a full gc to collect them. As for now stop-world pause could reach 6 seconds and I'd like to reduce it.

Average object size is 500k but could go up to 1 meg, so we are talking about 10000-20000 cached objects (actually byte arrays).

What is the best strategy for GC optimisation? I know that I can got off-heap, but it is kind a last resort solution.

Thank you!

Answer 1

10GB cache is not something you should do in the heap. Use ByteBuffers for caching. Object creation should not be that costly. This way there is no GC involved and you can manage everything by yourself.

For example if you implement a page cache in a Java Database Management System you would not create objects for it but use byte buffers or managed byte buffers or best direct byte buffers. You can learn more about those three here.

If you handle more then lets say a million objects at a time you will see the GC time share going up. I saw situations where we managed a huge number of nodes for data processing and it was really slow. We then switched to a direct byte buffer scheme and used even some additional technics we were able to fit more data in (objects cost 24bytes at least each) and stopped thinking about objects in first place. In the end we handled datas and not objects. This increased the performance by many times and we ware able to handle much more data then we expected.

After that we noticed it all fits a database and well that was the point we scraped everything.

So check out what direct buffers can do for you.

Answer 2

I routinely working with caching services holding 10-30 GiB of data in JVM heap. Concurent Mark Sweep (GC) algorithm can handle these cases pretty well, keeping max Stop-the-World pause around 100ms (though, absolute numbers depends on hardware).

You can find GC tuning check list for caching applications and heap sizing in my blog.

Here you can find more about Concurent Mark Sweep algorithm itself.