I am using kafka-node (https://github.com/SOHU-Co/kafka-node) consumer to retrieve data. I think the data which I get is compressed with SNAPPY. How do I decompress the data after I get it. I tried using node-snappy (https://github.com/kesla/node-snappy) to decompress the data, but it didn't work.
Is there any option in the library to set the compression to none?
Anyone used kafka-node library to get data from kafka..??
Thanks, chandu
I also encountered these exact problems. I found a solution, at last! You can use kafkacat ('like netcat for kafka') download here, which requires librdkafka. This enables you to interact with Kafka from the command line using the librdkafka C/C++ library. (no JVM required =D )
Now that we have those dependencies taken care of we can use this fun node.js repo: node-kafkacat
You'll find that there's likely enough documentation between those three libraries to get you started and unlike some of the other kafka-node modules on github, they seem to have been updated fairly recently.
I have only successfully installed on Linux and Mac so far but things are working great with our Apache/Java environment. I'm not the author of any of these packages, btw - just some guy who kept hoping your question would be answered over the last couple weeks.