trying to create topic with kafkajs in node js. using docker containers for Kafka and Zookeeper. getting the above error. what could be the reason? unfortunetly, googling it didn't help. i'm using windows 10 so working with WSL. Thanks
const {Kafka} = require("kafkajs")
run();
async function run(){
try
{
const kafka = new Kafka({
"clientId": "myapp",
"brokers" :["localhost:9092"]
})
const admin = kafka.admin();
console.log("Connecting.....")
await admin.connect()
console.log("Connected!")
//A-M, N-Z
await admin.createTopics({
"topics": [{
"topic" : "Users"
}]
})
console.log("Created Successfully!")
await admin.disconnect();
}
catch(ex)
{
console.error(`Something bad happened ${ex}`)
}
finally{
process.exit(0);
}
}
output is:
Connecting.....
Connected!
Something bad happened KafkaJSBrokerNotFound: Broker -1 not found in the cached metadata
docker-compose.yml:
services:
zookeeper:
image: zookeeper
hostname: zookeeper
ports:
- 2181:2181
kafka:
restart: always
image: ches/kafka
hostname: localhost
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: 109.253.189.218
ZOOKEEPER_IP: zookeeper
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
ALLOW_PLAINTEXT_LISTENER: 1
depends_on:
- zookeeper
DEBUG level logs: ....
Connected!
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.355Z","logger":"kafkajs","message":"[Connection] Request Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":3,"expectResponse":true,"size":23}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.357Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":3,"size":40,"data":{"brokers":[],"clusterId":"FsTkMSOPTlaLiIc5Qn65Fg","controllerId":-1,"topicMetadata":[]}}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.357Z","logger":"kafkajs","message":"[Connection] Request Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":4,"expectResponse":true,"size":23}
{"level":"DEBUG","timestamp":"2023-03-30T11:00:02.359Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 2)","broker":"localhost:9092","clientId":"myapp","correlationId":4,"size":40,"data":{"brokers":[],"clusterId":"FsTkMSOPTlaLiIc5Qn65Fg","controllerId":-1,"topicMetadata":[]}}
Something bad happened KafkaJSBrokerNotFound: Broker -1 not found in the cached metadata
Can you try using a maintained version of kafka (source):
# fill free to add more config
version: '2'
networks:
app-tier:
driver: bridge
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
networks:
- app-tier
kafka:
image: 'bitnami/kafka:latest'
networks:
- app-tier
myapp:
image: 'YOUR_APPLICATION_IMAGE'
networks:
- app-tier