spring-bootdockerelasticsearchjunit5camunda

@CamundaSpringProcessTest failing on MAC M4 15.2 elasticsearch container fails to start


I'm trying to run integration test between Camunda and my spring boot app. The problem is a JVM BUG that is preventing the containers created by @CamundaSpringProcessTest to start.

I have tryied to set the env in Junit runner, in gradle tasks.test, and using the Camunda test properties

Does someone knows how to tell the docker container created by the annotation to use an ENV variable? Does anyone knows a lower version of Camunda ,using a older jdk, we can use for testing?

We were able to solve the containers problem in local using ENV JAVA_TOOL_OPTIONS="-XX:UseSVE=0" but the issue stays in integration tests.

This is a simple integration test:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, classes = ComplianceApplication.class)
@AutoConfigureMockMvc
@CamundaSpringProcessTest
public class CamundaDebugRuleCreationIntTest {
    public static final String PATH = "/api/v1/rule/create";

    @Autowired
    private MockMvc mockMvc;

    @Autowired
    private ObjectMapper objectMapper;

    @Value("classpath:dmn/dmn_example.xml")
    private Resource body;

    @Test
    @DisplayName("GIVEN an incorrect payload to create a rule WHEN calling the endpoint to create it THEN the rule fails and an error is retrieved.")
    void shouldFailRuleCreationTest() throws Exception {
        CreateRuleDto createRuleDto = new CreateRuleDto(body.getContentAsString(Charset.defaultCharset()), null);

        String requestBody = objectMapper.writeValueAsString(createRuleDto);

        mockMvc.perform(post(PATH)
                        .contentType(MediaType.APPLICATION_JSON)
                        .content(requestBody))
                .andExpect(status().isInternalServerError());
    }
}

Error log

2025-02-21T16:38:10.374+01:00  INFO 17848 --- [    Test worker] tc.elasticsearch:8.13.4                  : Creating container for image: elasticsearch:8.13.4
2025-02-21T16:38:10.485+01:00  INFO 17848 --- [    Test worker] tc.elasticsearch:8.13.4                  : Container elasticsearch:8.13.4 is starting: ccc0131b22f58b06bd2cb6ad76a4307347c767391a4a36f5891044504a07b429
2025-02-21T16:38:10.609+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # A fatal error has been detected by the Java Runtime Environment:
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #  SIGILL (0x4) at pc=0x0000ffff6bf3fb5c, pid=8, tid=17
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # JRE version:  (21.0.2+13) (build )
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # Java VM: OpenJDK 64-Bit Server VM (21.0.2+13-58, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # Problematic frame:
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # j  java.lang.System.registerNatives()V+0 java.base@21.0.2
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2025-02-21T16:38:10.610+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.611+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # An error report file with more information is saved as:
2025-02-21T16:38:10.611+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # /usr/share/elasticsearch/hs_err_pid8.log
2025-02-21T16:38:10.611+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: [0.009s][warning][os] Loading hsdis library failed
2025-02-21T16:38:10.613+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.613+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # The crash happened outside the Java Virtual Machine in native code.
2025-02-21T16:38:10.613+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: # See problematic frame for where to report the bug.
2025-02-21T16:38:10.614+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDOUT: #
2025-02-21T16:38:10.614+01:00  INFO 17848 --- [tream-536931358] tc.elasticsearch                         : STDERR: /usr/local/bin/docker-entrypoint.sh: line 43:     8 Aborted                 ( elasticsearch-keystore create )
2025-02-21T16:39:10.632+01:00 ERROR 17848 --- [    Test worker] tc.elasticsearch:8.13.4                  : Could not start container

java.lang.IllegalStateException: Wait strategy failed. Container exited with code 134
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:346) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) ~[duct-tape-1.0.8.jar:na]
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:322) ~[testcontainers-1.20.4.jar:1.20.4]
    at io.camunda.process.test.impl.runtime.CamundaContainerRuntime.start(CamundaContainerRuntime.java:123) ~[camunda-process-test-java-8.6.7.jar:8.6.7]
    at io.camunda.process.test.api.CamundaProcessTestExecutionListener.beforeTestMethod(CamundaProcessTestExecutionListener.java:77) ~[camunda-process-test-spring-8.6.7.jar:8.6.7]
    at org.springframework.test.context.TestContextManager.beforeTestMethod(TestContextManager.java:320) ~[spring-test-6.2.1.jar:6.2.1]
    at org.springframework.test.context.junit.jupiter.SpringExtension.beforeEach(SpringExtension.java:237) ~[spring-test-6.2.1.jar:6.2.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachCallbacks$3(TestMethodTestDescriptor.java:166) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$7(TestMethodTestDescriptor.java:202) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:202) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachCallbacks(TestMethodTestDescriptor.java:165) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:132) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68) ~[junit-jupiter-engine-5.11.4.jar:5.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:156) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) ~[junit-platform-engine-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) ~[junit-platform-launcher-1.11.4.jar:1.11.4]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:124) ~[na:na]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:99) ~[na:na]
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:94) ~[na:na]
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63) ~[na:na]
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) ~[na:na]
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) ~[na:na]
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) ~[na:na]
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92) ~[na:na]
    at jdk.proxy1/jdk.proxy1.$Proxy4.stop(Unknown Source) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:200) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:132) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:103) ~[na:na]
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63) ~[na:na]
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) ~[na:na]
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:121) ~[na:na]
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) ~[na:na]
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) ~[gradle-worker.jar:na]
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74) ~[gradle-worker.jar:na]
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for log output matching '.*("message":\s?"started[\s?|"].*|] started
$)'
    at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:47) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:909) ~[testcontainers-1.20.4.jar:1.20.4]
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:492) ~[testcontainers-1.20.4.jar:1.20.4]
    ... 75 common frames omitted

2025-02-21T16:39:10.674+01:00 ERROR 17848 --- [    Test worker] tc.elasticsearch:8.13.4                  : Log output from the failed container:
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x0000ffff6bf3fb5c, pid=8, tid=17
#
# JRE version:  (21.0.2+13) (build )
# Java VM: OpenJDK 64-Bit Server VM (21.0.2+13-58, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
# Problematic frame:
# j  java.lang.System.registerNatives()V+0 java.base@21.0.2
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /usr/share/elasticsearch/hs_err_pid8.log
[0.009s][warning][os] Loading hsdis library failed
#
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
/usr/local/bin/docker-entrypoint.sh: line 43:     8 Aborted                 ( elasticsearch-keystore create )

We are using jdk 21, spring boot 3.4.1, camunda 8.6.7 and gradle


Solution

  • I made a PR to camunda. If you need to use it You have to add these property in your project:

    io:
      camunda:
        process:
          test:
            camundaEnvVars:
              JAVA_TOOL_OPTIONS: -XX:UseSVE=0
            elasticsearchEnvVars:
              ES_JAVA_OPTS: -XX:UseSVE=0
              CLI_JAVA_OPTS: -XX:UseSVE=0
            connectorsEnvVars:
              JAVA_TOOL_OPTIONS: -XX:UseSVE=0
    

    and deploy a temporary version on your nexus/jfrog