Is java native memory used in Log4j2 ? I found many log in native memory instead of heap

Question

We have a java app started by running Main function, we also start embedded jetty in springboot as webcontainer. I found java head reached max size , but heap usage is low and java run out native memory killed by OS

There are many 64MB memory in pmap. I dump some memory blocks and found many log in it. The log time is various, it seems log still in memory even days passed. for exmaple

2019-03-23T05:50:46,851 661258664 [xxxxxx] INFO
2019-03-27T06:00:12,040 1029308155 [xxxxxxxx] INFO .........

We use log4j2 and slf4j as log tool. Logs was printed by jetty thread which name start with 'qtp'. We set immediateFlush="false" and SizeBasedTriggeringPolicy size="10MB"

I don't know why the log content are NOT in heap. Is log4j2 using java native memory when cache or write log ? Is it possible log4j2 memory leak?

Answer 1

I suspect what you are seeing is a side effect of a little-known feature of Log4j 2: garbage-free logging.

Since log4j 2.6, the library takes extreme care to not allocate any temporary objects during steady-state logging (there is some allocation of temporary objects during initialization and configuration).

Other logging libraries that don’t have this feature will typically allocate many LogEvent objects, as well as Strings, char[] arrays and byte[] arrays. These are typically very short-lived and quickly collected, but significant logging will fill up the Young Generation and contribute to promotions.

On the HotSpot JVM even minor collections are stop-the-world collections with all current GC algorithms: Serial, Parallel, Concurrent, CMS, and G1. (This may become less impactful with Shenandoah and ZGC, but these collectors are still experimental as of this writing. Also note that Zing/Azul has a pauseless collector, however that work has not been contributed to OpenJDK.)

Log4j 2 sidesteps this by avoiding allocation and reusing memory where possible.