javaspring-bootmemorymemory-leaksoverflow

SpringBoot application memory consistently exceeds JVM configured heap size


Spring Boot application memory usage continues to exceed the JVM configured heap size without overflow errors, RES usage is approaching 5G when viewed with the top command.

My application is deployed on Linux, the frameworks used in the project are spring-rabbit (version 2.3.11) with kafka-clients (2.7.1) and spring-kafka (2.7.8). The project has a single HTTP interface, and the code inside the interface is used in the following way:

private final KafkaTemplate<String, String> kafkaTemplate;
private final RabbitTemplate rabbitTemplate;
rabbitTemplate.convertAndSend("cloud", "ctrlLog", data);
kafkaTemplate.send(BIG_DATA_TOPIC, data);

JVM Configuration:

Tried by myself, -XX:NativeMemoryTracking=detail export file analysis and dump file analysis, but did not find the problem, after a few days through the jcmd pid VM.native_memory detail.diff scale=MB found to be

[0x00007f32dc1cf5ea] JNIHandleBlock::allocate_block(Thread*)+0xaa
[0x00007f32dc163dda] JavaCallWrapper::JavaCallWrapper(methodHandle, Handle, JavaValue*, Thread*)+0x6a
[0x00007f32dc166840] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x8f0
[0x00007f32dc1debf4] JVM_DoPrivileged+0x4f4
                             (malloc=1049MB type=Internal +456MB #3526800 +1531841)

But I don't really understand where the leak occurs.

Documents exported in connection with your own analysis of the problem.


Solution

  • English is not my native language; please excuse typing errors.

    The current problem has been solved.

    The leak has been resolved with a profiler flame map.

    The cause of the leak was a JSON utility class that kept loading the class safely when doing toString.