What I would like to achieve:
Run a kafka consumer with native / aot / graalvm
Issue:
When running the application, I am facing this 100% reproducible issue:
Caused by: org.apache.kafka.common.KafkaException: Received exception when fetching the next record from my-topic-1. If needed, please seek past the record to continue consumption.
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1676)
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1900(Fetcher.java:1497)
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:717)
at org.apache.kafka.clients.consumer.internals.Fetcher.collectFetch(Fetcher.java:683)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1314)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1243)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216)
at reactor.kafka.receiver.internals.ConsumerEventLoop$PollEvent.run(ConsumerEventLoop.java:357)
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
at java.base@17.0.7/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base@17.0.7/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base@17.0.7/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)
Caused by: org.apache.kafka.common.KafkaException: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux and os.arch=x86_64
at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:46)
at org.apache.kafka.common.record.CompressionType$3.wrapForInput(CompressionType.java:94)
at org.apache.kafka.common.record.DefaultRecordBatch.recordInputStream(DefaultRecordBatch.java:276)
at org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:280)
at org.apache.kafka.common.record.DefaultRecordBatch.streamingIterator(DefaultRecordBatch.java:364)
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.nextFetchedRecord(Fetcher.java:1619)
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1656)
... 16 common frames omitted
Caused by: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Linux and os.arch=x86_64
at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:343)
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:177)
at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:155)
at org.xerial.snappy.Snappy.init(Snappy.java:70)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
at org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:145)
at org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:99)
at org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:59)
at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:44)
... 22 common frames omitted
What I tried:
In my dependencies, I do have the most recent Snappy:
<dependency>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
<version>1.1.9.1</version>
</dependency>
On my machine, Ubuntu, I do have Snappy installed.
root@dlp:~# apt -y install snapd
root@dlp:~# systemctl enable snapd.service snapd.socket
root@dlp:~# source /etc/profile.d/apps-bin-path.sh
Question:
How to fix this exception?
I do have Snappy installed.
Snap / snapd is not the same as Snappy .
You should install the latter with libsnappy-dev
, as that is the native libraries that the error is referring to. The Java dependency requires those.
If you don't need Snap applications, then you can uninstall snapd