I'm deploying Apache Cassandra on a managed RHEL8 instance. I can install and start Cassandra successfully if I remount /tmp without the noexec flag. Overnight the noexec flag is reinstated by company security policies.
If the noexec flag is present, Cassandra fails to start with the error
ERROR [main] 2024-06-24 10:19:18,156 NativeLibraryLinux.java:65 - Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /tmp/jna8661137960014770832.tmp: /tmp/jna8661137960014770832.tmp: failed to map segment from shared object
The solution should be to change the tmp directory for Cassandra. I've set the following in cassandra-env.sh:
JVM_OPTS="$JVM_OPTS -Djna.tmpdir=/var/lib/cassandra/tmp"
JVM_OPTS="$JVM_OPTS -Djava.io.tmpdir=/var/lib/cassandra/tmp"`
I get a different error, earlier in the Cassandra startup now:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f80d11e5152, pid=2277605, tid=2277607
#
# JRE version: OpenJDK Runtime Environment (Red_Hat-11.0.23.0.9-2) (11.0.23+9) (build 11.0.23+9-LTS)
# Java VM: OpenJDK 64-Bit Server VM (Red_Hat-11.0.23.0.9-2) (11.0.23+9-LTS, mixed mode, sharing, tiered, compressed oops, concurrent mark sweep gc, linux-amd64)
# Problematic frame:
# C [jna10993927332404549884.tmp+0x13152] ffi_prep_closure_loc+0x32`
One can see above that the fatal error occurred in the jna library.
Inspecting the error report file, it looks like linking errors:
Internal exceptions (20 events):
Event: 2.449 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x00000006082a84f0}: 'long java.lang.invoke.DirectMethodHandle$Holder.invokeVirtual(java.lang.Object, java.lang.Object)'> (0x00000006082a84f0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.456 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060831f288}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object)'> (0x000000060831f288) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.485 Thread 0x00007f810002e050 Exception <a 'java/lang/reflect/InvocationTargetException'{0x00000006085bb768}> (0x00000006085bb768) thrown at [src/hotspot/share/runtime/reflection.cpp, line 1245]
Event: 2.487 Thread 0x00007f810002e050 Exception <a 'java/lang/reflect/InvocationTargetException'{0x00000006085d34f8}> (0x00000006085d34f8) thrown at [src/hotspot/share/runtime/reflection.cpp, line 1245]
Event: 2.545 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x00000006087476a0}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x00000006087476a0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.546 Thread 0x00007f810002e050 Exception <a 'java/lang/IncompatibleClassChangeError'{0x00000006087504d0}: Found class java.lang.Object, but interface was expected> (0x00000006087504d0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 839]
Event: 2.546 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x0000000608755740}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object)'> (0x0000000608755740) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.717 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x0000000609606ba8}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x0000000609606ba8) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.726 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x00000006096ee610}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object)'> (0x00000006096ee610) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.817 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060b8ed080}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeVirtual(java.lang.Object, java.lang.Object)'> (0x000000060b8ed080) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.817 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060b8f57e0}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object)'> (0x000000060b8f57e0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.817 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060b8f81a0}: 'void java.lang.invoke.Invokers$Holder.invoke_MT(java.lang.Object, java.lang.Object)'> (0x000000060b8f81a0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.839 Thread 0x00007f810002e050 Exception <a 'java/lang/ClassNotFoundException'{0x000000060b968f90}: javax/management/remote/rmi/RMIServerImpl_Skel> (0x000000060b968f90) thrown at [src/hotspot/share/classfile/systemDictionary.cpp, line 230]
Event: 2.855 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060ba07d50}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, int, long)'> (0x000000060ba07d50) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.888 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060bbf30c0}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, long, java.lang.Object)'> (0x000000060bbf30c0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.889 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060bbfefc0}: 'java.lang.Object java.lang.invoke.Invokers$Holder.linkToTargetMethod(java.lang.Object, long, java.lang.Object)'> (0x000000060bbfefc0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.890 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060bc05498}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object, int)'> (0x000000060bc05498) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.890 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060bc08db0}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, int)'> (0x000000060bc08db0) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.890 Thread 0x00007f810002e050 Exception <a 'java/lang/NoSuchMethodError'{0x000000060bc0be18}: 'java.lang.Object java.lang.invoke.Invokers$Holder.linkToTargetMethod(java.lang.Object, java.lang.Object, int, java.lang.Object)'> (0x000000060bc0be18) thrown at [src/hotspot/share/interpreter/linkResolver.cpp, line 773]
Event: 2.895 Thread 0x00007f810002e050 Implicit null exception at 0x00007f80f02ad984 to 0x00007f80f02addec
I'm using the bundled jna JAR with Apache Cassandra 4.1.5-1 (jna JAR: jna-5.9.0.jar).
I've tried uninstalling and reinstalling both Cassandra and Java. The Java version installed is the determined dependency by yum: java-1.8.0-openjdk-headless.x86_64 1:1.8.0.412.b08-2.el8 @rhel-8-for-x86_64-appstream-rpms
This problem was caused by conflicting Java versions (both JDK 8).
I removed Cassandra, which removed the JDK 8 dependency too, then removed the other JDK 8 installation. Then I installed JDK 11 and re-installed Cassandra. There was now only a single JDK 11 installation, and the only installed version of Java. Cassandra runs without issues.