The ticket said "memory leak". The heap graph was flat. Both were true, and the second one was the clue.
The heap is a minority shareholder
A JVM in a container spends memory on metaspace, code cache, thread stacks, GC structures and direct buffers, none of which appear in a heap dump. Native Memory Tracking is the only honest accounting, and it is off by default.
- Set
-XX:MaxRAMPercentagerather than a fixed-Xmx, so the limit follows the pod. - Every thread is a stack. A thread-per-connection server with a 1MB default is a memory plan, not an accident.
- Direct buffers live outside the heap and are freed when the referring object is collected, which may be much later than you assume.
We found ours in a Netty buffer pool sized for a machine four times larger than the pod. The heap was never the problem.



Tags:
3 Comments
Leave a comment
Sign in to leave a comment.
Signing in is unavailable right now.







Sanne de Vries
May 11, 2024MaxRAMPercentage is the single setting I wish every Java image shipped with by default.
Omar Haddad
May 12, 2024هل تنصح بتشغيل Native Memory Tracking في الإنتاج دائماً؟
Muhammed Senussi
AuthorMay 12, 2024بمستوى summary نعم، فتكلفته حوالي 5% وقد وفّر علينا أياماً من التخمين. أما مستوى detail فأشغّله فقط أثناء التحقيق.