We recently upgraded from Java 8 to Java 21 and noticed a significant increase in memory consumption in our Java applications. Our applications are deployed in containers based on Alpine OS, and we use the Eclipse Temurin Build.
Reproducing the Issue
I have created a small project that replicates this issue, which is available at: https://github.com/RaghuChandrasekaran/java21-memory-usage.
The docker stats report the following memory usage for the Java process when the heap values are -Xms256m -Xmx512m with the memory limit of 1 G. Memory usage jumps from 231 MiB to 314, a third higher.

Analysis To investigate the issue, I enabled Native Memory Tracking (NMT), and the output is provided below.
Java 8
Native Memory Tracking:
Total: reserved=1929874KB, committed=393190KB
- Java Heap (reserved=524288KB, committed=262208KB)
(mmap: reserved=524288KB, committed=262208KB)
- Class (reserved=1099014KB, committed=57566KB)
(classes #10367)
(malloc=1286KB #10537)
(mmap: reserved=1097728KB, committed=56280KB)
- Thread (reserved=31968KB, committed=31968KB)
(thread #32)
(stack: reserved=31829KB, committed=31829KB)
(malloc=106KB #186)
(arena=32KB #58)
- Code (reserved=253118KB, committed=20810KB)
(malloc=3518KB #5282)
(mmap: reserved=249600KB, committed=17292KB)
- GC (reserved=1738KB, committed=890KB)
(malloc=26KB #174)
(mmap: reserved=1712KB, committed=864KB)
- Compiler (reserved=159KB, committed=159KB)
(malloc=24KB #469)
(arena=135KB #7)
- Internal (reserved=1844KB, committed=1844KB)
(malloc=1812KB #12019)
(mmap: reserved=32KB, committed=32KB)
- Symbol (reserved=15183KB, committed=15183KB)
(malloc=11915KB #122017)
(arena=3268KB #1)
- Native Memory Tracking (reserved=2366KB, committed=2366KB)
(malloc=7KB #90)
(tracking overhead=2359KB)
- Arena Chunk (reserved=196KB, committed=196KB)
(malloc=196KB)
Java 21
Native Memory Tracking:
Total: reserved=2044186KB, committed=424038KB
malloc: 42596KB #337605
mmap: reserved=2001590KB, committed=381442KB
- Java Heap (reserved=524288KB, committed=262208KB)
(mmap: reserved=524288KB, committed=262208KB)
- Class (reserved=1050068KB, committed=13012KB)
(classes #17813)
( instance classes #16553, array classes #1260)
(malloc=1492KB #45190) (peak=1496KB #45058)
(mmap: reserved=1048576KB, committed=11520KB)
( Metadata: )
( reserved=131072KB, committed=69376KB)
( used=68992KB)
( waste=384KB =0.55%)
( Class space:)
( reserved=1048576KB, committed=11520KB)
( used=11130KB)
( waste=390KB =3.38%)
- Thread (reserved=31927KB, committed=2831KB)
(thread #32)
(stack: reserved=31826KB, committed=2730KB)
(malloc=66KB #191) (peak=79KB #207)
(arena=34KB #60) (peak=2322KB #38)
- Code (reserved=249888KB, committed=24908KB)
(malloc=2200KB #9067) (at peak)
(mmap: reserved=247688KB, committed=22708KB)
- GC (reserved=1734KB, committed=882KB)
(malloc=22KB #80) (peak=250KB #137)
(mmap: reserved=1712KB, committed=860KB)
- Compiler (reserved=238KB, committed=238KB)
(malloc=74KB #835) (peak=117KB #857)
(arena=164KB #4) (peak=53061KB #30)
- Internal (reserved=863KB, committed=863KB)
(malloc=827KB #25496) (peak=835KB #25298)
(mmap: reserved=36KB, committed=36KB)
- Other (reserved=24KB, committed=24KB)
(malloc=24KB #3) (peak=58KB #5)
- Symbol (reserved=30116KB, committed=30116KB)
(malloc=28254KB #234525) (at peak)
(arena=1862KB #1) (at peak)
- Native Memory Tracking (reserved=5362KB, committed=5362KB)
(malloc=87KB #1569) (peak=87KB #1568)
(tracking overhead=5275KB)
- Shared class space (reserved=16384KB, committed=11996KB, readonly=0KB)
(mmap: reserved=16384KB, committed=11996KB)
- Arena Chunk (reserved=2KB, committed=2KB)
(malloc=2KB #116) (peak=55748KB #1365)
- Module (reserved=127KB, committed=127KB)
(malloc=127KB #3158) (at peak)
- Safepoint (reserved=8KB, committed=8KB)
(mmap: reserved=8KB, committed=8KB)
- Synchronization (reserved=1751KB, committed=1751KB)
(malloc=1751KB #17227) (at peak)
- Serviceability (reserved=17KB, committed=17KB)
(malloc=17KB #9) (peak=17KB #11)
- Metaspace (reserved=131387KB, committed=69691KB)
(malloc=315KB #118) (at peak)
(mmap: reserved=131072KB, committed=69376KB)
- String Deduplication (reserved=1KB, committed=1KB)
(malloc=1KB #8) (at peak)

The possible suspects that I had were,
But testing with the spring-pet-clinic project which doesn't use Netty ruled out #1 and testing in Ubuntu Base OS ruled out #2.
What additional steps can I take to further diagnose the cause of this behavior?
Update on Oct 2nd:
As per @matt's request, I tried with a Simple HTTP Server in Native Java without any dependency and can reproduce the same behavior (Java 21 using slightly higher memory than Java 8). In this case, the memory usage is not as high as in the Spring Pet Clinic. Changes are updated in the https://github.com/RaghuChandrasekaran/java21-memory-usage repo.

I am not facing same issues.
Then I ran both docker's with the commands you specified on github repository
Executed docker stats

java8 results:
Connected to remote JVM
JVM response code = 0
Native Memory Tracking:
Total: reserved=1930424KB, committed=396664KB
- Java Heap (reserved=524288KB, committed=262208KB)
(mmap: reserved=524288KB, committed=262208KB)
- Class (reserved=1099019KB, committed=57571KB)
(classes #10364)
(malloc=1291KB #10885)
(mmap: reserved=1097728KB, committed=56280KB)
- Thread (reserved=31968KB, committed=31968KB)
(thread #32)
(stack: reserved=31829KB, committed=31829KB)
(malloc=106KB #186)
(arena=32KB #58)
- Code (reserved=253651KB, committed=24267KB)
(malloc=4051KB #5485)
(mmap: reserved=249600KB, committed=20216KB)
- GC (reserved=1738KB, committed=890KB)
(malloc=26KB #174)
(mmap: reserved=1712KB, committed=864KB)
- Compiler (reserved=159KB, committed=159KB)
(malloc=24KB #504)
(arena=135KB #7)
- Internal (reserved=1841KB, committed=1841KB)
(malloc=1809KB #11987)
(mmap: reserved=32KB, committed=32KB)
- Symbol (reserved=15190KB, committed=15190KB)
(malloc=11921KB #121974)
(arena=3268KB #1)
- Native Memory Tracking (reserved=2374KB, committed=2374KB)
(malloc=7KB #90)
(tracking overhead=2367KB)
- Arena Chunk (reserved=196KB, committed=196KB)
(malloc=196KB)
java21 results:
Native Memory Tracking:
(Omitting categories weighting less than 1KB)
Total: reserved=2075752KB, committed=427820KB
malloc: 42667KB #338045
mmap: reserved=2033085KB, committed=385153KB
- Java Heap (reserved=524288KB, committed=262208KB)
(mmap: reserved=524288KB, committed=262208KB)
- Class (reserved=1050065KB, committed=13009KB)
(classes #17806)
( instance classes #16545, array classes #1261)
(malloc=1489KB #45243) (peak=1492KB #45058)
(mmap: reserved=1048576KB, committed=11520KB)
( Metadata: )
( reserved=131072KB, committed=69568KB)
( used=69098KB)
( waste=470KB =0.68%)
( Class space:)
( reserved=1048576KB, committed=11520KB)
( used=11126KB)
( waste=394KB =3.42%)
- Thread (reserved=63422KB, committed=4674KB)
(thread #32)
(stack: reserved=63321KB, committed=4573KB)
(malloc=66KB #191) (peak=79KB #207)
(arena=34KB #60) (peak=2322KB #38)
- Code (reserved=249970KB, committed=26666KB)
(malloc=2282KB #9141) (at peak)
(mmap: reserved=247688KB, committed=24384KB)
- GC (reserved=1734KB, committed=882KB)
(malloc=22KB #80) (peak=250KB #137)
(mmap: reserved=1712KB, committed=860KB)
- Compiler (reserved=211KB, committed=211KB)
(malloc=47KB #819) (peak=64KB #843)
(arena=164KB #4) (peak=43583KB #28)
- Internal (reserved=865KB, committed=865KB)
(malloc=829KB #25631) (peak=835KB #25331)
(mmap: reserved=36KB, committed=36KB)
- Other (reserved=28KB, committed=28KB)
(malloc=28KB #3) (peak=62KB #5)
- Symbol (reserved=30121KB, committed=30121KB)
(malloc=28259KB #234655) (at peak)
(arena=1862KB #1) (at peak)
- Native Memory Tracking (reserved=5374KB, committed=5374KB)
(malloc=92KB #1650) (at peak)
(tracking overhead=5282KB)
- Shared class space (reserved=16384KB, committed=11996KB, readonly=0KB)
(mmap: reserved=16384KB, committed=11996KB)
- Arena Chunk (reserved=2KB, committed=2KB)
(malloc=2KB #116) (peak=45340KB #1149)
- Module (reserved=128KB, committed=128KB)
(malloc=128KB #3160) (at peak)
- Safepoint (reserved=8KB, committed=8KB)
(mmap: reserved=8KB, committed=8KB)
- Synchronization (reserved=1748KB, committed=1748KB)
(malloc=1748KB #17209) (at peak)
- Serviceability (reserved=17KB, committed=17KB)
(malloc=17KB #9) (peak=17KB #11)
- Metaspace (reserved=131386KB, committed=69882KB)
(malloc=314KB #117) (at peak)
(mmap: reserved=131072KB, committed=69568KB)
- String Deduplication (reserved=1KB, committed=1KB)
(malloc=1KB #8) (at peak)
| Category | Java 8 - Committed Memory in KB | Java 21 - Committed Memory in KB | Diff (Java 21 - Java 8) |
|---|---|---|---|
| Heap | 262208 | 262208 | 0 |
| Class | 57571 | 13009 | -44562 |
| Thread | 31968 | 4674 | -27294 |
| Code | 24267 | 26666 | 2399 |
| GC | 890 | 882 | -8 |
| Compiler | 159 | 211 | 52 |
| Internal | 1841 | 865 | -976 |
| Symbol | 15190 | 30121 | 14931 |
| Native Memory Tracking | 2374 | 5374 | 3000 |
| Arena Chunk | 196 | 2 | -194 |
| Other | 0 | 28 | 28 |
| Shared Class Space | 0 | 11996 | 11996 |
| Module | 0 | 128 | 128 |
| SafePoint | 0 | 8 | 8 |
| Synchronization | 0 | 1748 | 1748 |
| Serviceability | 0 | 17 | 17 |
| Metaspace | 0 | 69882 | 69882 |
| String Deduplication | 0 | 1 | 1 |
I executed a heapdump and it shows java21 consuming less memory than java8
JAVA21

JAVA8

Why does Native Memory Tracking (NMT) report Java21 as consuming more memory than Java8? To be transparent, I do not know. Probably someone has the answer to that from Java8 to Java21 many changes have been done even in the reporting and capturing data so I guess Java21 is capturing more data... Java8 reports as classes while Java21 reports it as Metaspace
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With