jvm-hotspotmemory-fragmentation

How do I measure fragmentation in Hotspot's Metaspace?


I'm looking into debugging an "OutOfMemoryError: Metaspace" error in my application. Right before the OOME I see the following in the gc logs:

{Heap before GC invocations=6104 (full 39):
 par new generation   total 943744K, used 0K [...)
  eden space 838912K,   0% used [...)
  from space 104832K,   0% used [...)
  to   space 104832K,   0% used [...)
 concurrent mark-sweep generation total 2097152K, used 624109K [...)
 Metaspace       used 352638K, capacity 487488K, committed 786432K, reserved 1775616K
  class space    used 36291K, capacity 40194K, committed 59988K, reserved 1048576K
2015-08-11T20:34:13.303+0000: 105892.129: [Full GC (Last ditch collection) 105892.129: [CMS: 624109K->623387K(2097152K), 3.4208207 secs] 624109K->623387K(3040896K), [Metaspace: 352638K->352638K(1775616K)], 3.4215100 secs] [Times: user=3.42 sys=0.00, real=3.42 secs] 
Heap after GC invocations=6105 (full 40):
 par new generation   total 943744K, used 0K [...)
  eden space 838912K,   0% used [...)
  from space 104832K,   0% used [...)
  to   space 104832K,   0% used [...)
 concurrent mark-sweep generation total 2097152K, used 623387K [...)
 Metaspace       used 352638K, capacity 487488K, committed 786432K, reserved 1775616K
  class space    used 36291K, capacity 40194K, committed 59988K, reserved 1048576K
}

From what I can see, Metaspace capacity isn't even nearing the committed size (in this case, -XX:MaxMetaspaceSize=768m). So I suspect fragmentation of Metaspace causing the allocator to fail to find a new chunk for the new classloader.

I'm aware of -XX:PrintFLSStatistics but that only covers CMS, not native memory.

So my question is: is there a debugging help similar to PrintFLSStatistics available for Hotspot's native memory?

This is using Java HotSpot(TM) 64-Bit Server VM (25.45-b02) for linux-amd64 JRE (1.8.0_45-b14).


Solution

  • I've just looked into the implementation of the Metaspace in HotSpot. The Metaspace is divided into chunks and managed using a freelist. So fragmentation is indeed a possible reason for your problem.

    I've also looked through the flags of the HotSpot VM (-XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal), there is no flag in the release version.

    However, there is a dump() method in the Metaspace class which seems to be triggered by setting the -XX:+TraceMetadataChunkAllocation flag. There is also the -XX:+TraceMetavirtualspaceAllocation which is sounding to be of interest for you. However, those are "develop" flags, meaning you need a debug version of the VM.