Diagnosing OOMKilled in Spring Boot Container: Step-by-Step

Exit 137 means the kernel killed the container because it exceeded its cgroup memory limit — the JVM had no chance to react.

Step 1 — confirm OOMKilled, not some other 137 cause

• kubectl describe pod shows Reason: OOMKilled and Exit Code: 137 in the container's last state — confirm both before assuming OOM
• For Docker, docker inspect --format '{{.State.OOMKilled}}' returns true; on the host, dmesg | grep killed shows the kernel kill log

Step 2 — read the memory history before the kill

• Without metrics from the previous incarnation, you cannot diagnose root cause; if Actuator + Prometheus or Datadog is not in place, install it first and wait for the next kill
• Plot container_memory_working_set_bytes (RSS the kernel counts) alongside jvm_memory_used (heap) and jvm_memory_used{area=nonheap} over the 30 minutes preceding the kill — the shape of the curves tells you the cause

Step 3 — differentiate heap leak vs non-heap leak vs under-sized container

• Heap rising monotonically → heap leak; take a heap dump with jcmd 1 GC.heap_dump /tmp/heap.hprof, copy out, analyze with Eclipse MAT's Leak Suspects report — usually finds an unbounded cache, leaked listener, or escaped ThreadLocal
• Heap stable but Metaspace or direct buffers growing → classloader or NIO leak; enable -XX:NativeMemoryTracking=summary and jcmd 1 VM.native_memory summary to see which non-heap category is growing
• Both heap and non-heap stable but RSS hits limit fast → the container is just under-sized; tune MaxRAMPercentage from 75% down to 60% or raise the limit

Step 4 — fix the root cause, not the symptom

• For a heap leak, the fix is in the application code (add eviction, fix the cleanup); for a non-heap leak, the fix may be tuning a JVM flag (MaxMetaspaceSize, MaxDirectMemorySize) or fixing the leak
• Raising the container limit without finding the leak just delays the next OOMKill until the leak grows into the new limit — leaks have nowhere to go but up

Long-term hardening

• Always run with Actuator + Prometheus exporters so the next incident has metrics; add -XX:+HeapDumpOnOutOfMemoryError with a PersistentVolume mount so in-JVM OOMs leave a dump behind
• Set memory requests = limits for the QoS Guaranteed class, ensuring the pod is evicted last under node pressure