javaapache-kafkasnappy

Caused by: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux and os.arch=x86_64


What I would like to achieve:

Run a kafka consumer with native / aot / graalvm

Issue:

When running the application, I am facing this 100% reproducible issue:

Caused by: org.apache.kafka.common.KafkaException: Received exception when fetching the next record from my-topic-1. If needed, please seek past the record to continue consumption.
    at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1676)
    at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1900(Fetcher.java:1497)
    at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:717)
    at org.apache.kafka.clients.consumer.internals.Fetcher.collectFetch(Fetcher.java:683)
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1314)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1243)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216)
    at reactor.kafka.receiver.internals.ConsumerEventLoop$PollEvent.run(ConsumerEventLoop.java:357)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
    at java.base@17.0.7/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base@17.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
    at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
    at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)
Caused by: org.apache.kafka.common.KafkaException: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux and os.arch=x86_64
    at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:46)
    at org.apache.kafka.common.record.CompressionType$3.wrapForInput(CompressionType.java:94)
    at org.apache.kafka.common.record.DefaultRecordBatch.recordInputStream(DefaultRecordBatch.java:276)
    at org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:280)
    at org.apache.kafka.common.record.DefaultRecordBatch.streamingIterator(DefaultRecordBatch.java:364)
    at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.nextFetchedRecord(Fetcher.java:1619)
    at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1656)
    ... 16 common frames omitted
Caused by: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux and os.arch=x86_64
    at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:343)
    at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:177)
    at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:155)
    at org.xerial.snappy.Snappy.init(Snappy.java:70)
    at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
    at org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:145)
    at org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:99)
    at org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:59)
    at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:44)
    ... 22 common frames omitted

What I tried:

In my dependencies, I do have the most recent Snappy:

        <dependency>
            <groupId>org.xerial.snappy</groupId>
            <artifactId>snappy-java</artifactId>
            <version>1.1.9.1</version>
        </dependency>

On my machine, Ubuntu, I do have Snappy installed.

root@dlp:~# apt -y install snapd
root@dlp:~# systemctl  enable snapd.service snapd.socket
root@dlp:~# source /etc/profile.d/apps-bin-path.sh

Question:

How to fix this exception?


Solution

  • I do have Snappy installed.

    Snap / snapd is not the same as Snappy .

    You should install the latter with libsnappy-dev, as that is the native libraries that the error is referring to. The Java dependency requires those.

    If you don't need Snap applications, then you can uninstall snapd