javaapache-kafkaspring-kafkajava-17snappy

SpringBoot Kafka + openjdk 17: org.xerial.snappy.SnappyError: [UNSUPPORTED_PLATFORM] pure-java snappy requires access to java.nio.Buffer raw address


On a very simple Kafka consumer app from tutorial website: https://www.baeldung.com/spring-kafka

But once containerized with openJDK 17, this issue is 100% reprodicible:

ERROR 1 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : Consumer exception

java.lang.IllegalStateException: This error handler cannot process 'org.apache.kafka.common.KafkaException's; no record information is available
        at org.springframework.kafka.listener.DefaultErrorHandler.handleOtherException(DefaultErrorHandler.java:204) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.handleConsumerException(KafkaMessageListenerContainer.java:1955) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1390) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:831) ~[na:na]
Caused by: org.apache.kafka.common.KafkaException: Received exception when fetching the next record from thetopic. If needed, please seek past the record to continue consumption.
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1673) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1900(Fetcher.java:1494) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:716) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher.collectFetch(Fetcher.java:682) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1318) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1247) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1220) ~[kafka-clients-3.3.1.jar!/:na]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollConsumer(KafkaMessageListenerContainer.java:1670) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1645) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1446) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1338) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        ... 2 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: org.xerial.snappy.SnappyError: [UNSUPPORTED_PLATFORM] pure-java snappy requires access to java.nio.Buffer raw address field
        at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:46) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.CompressionType$3.wrapForInput(CompressionType.java:94) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.recordInputStream(DefaultRecordBatch.java:276) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:280) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.streamingIterator(DefaultRecordBatch.java:364) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.nextFetchedRecord(Fetcher.java:1616) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1653) ~[kafka-clients-3.3.1.jar!/:na]
        ... 12 common frames omitted
Caused by: org.xerial.snappy.SnappyError: [UNSUPPORTED_PLATFORM] pure-java snappy requires access to java.nio.Buffer raw address field
        at org.xerial.snappy.pure.UnsafeUtil.<clinit>(UnsafeUtil.java:49) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.getUnsignedByteSafe(SnappyRawDecompressor.java:323) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.readUncompressedLength(SnappyRawDecompressor.java:288) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.getUncompressedLength(SnappyRawDecompressor.java:42) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.PureJavaSnappy.uncompressedLength(PureJavaSnappy.java:238) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:638) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:145) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:99) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:59) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:44) ~[kafka-clients-3.3.1.jar!/:na]
        ... 18 common frames omitted

The thing is, with other JDK base images, or not dockerized environment, this issue is not reproducible.

I tried pulling this dependency, no luck.

       <dependency>
            <groupId>org.xerial.snappy</groupId>
            <artifactId>snappy-java</artifactId>
            <version>1.1.8.4</version>
        </dependency>

Solution

  • Add bellow vm options solve my issue: Add bellow vm options solve my issue:

    --add-opens=java.base/java.nio=ALL-UNNAMED
    --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
    --add-opens=java.base/sun.nio.cs=ALL-UNNAMED