I'm encountering an issue:
Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value for configuration message.max.bytes: Not a number of type INT
when trying to run bitnami image kafka broker inside Docker. It has appeared when I had switched to Macbook Pro M3. On Intel Macbook Pro it had been working smoothly out of box. What should I do to make this Kafka instance work? I have installed Java via brew but it didn't help. My docker-compose file look like this:
services:
kafka:
image: &kafka-image bitnami/kafka:latest
ports:
- ${KAFKA_BROKER_PORT_1}:${KAFKA_BROKER_PORT_1}
environment:
- KAFKA_CFG_NODE_ID=0
- KAFKA_CFG_PROCESS_ROLES=controller,broker
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:${KAFKA_BROKER_PORT_1}
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,EXTERNAL://${KAFKA_BROKER_SERVER_1}:${KAFKA_BROKER_PORT_1}
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT
- KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093
- KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
- KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
- KAFKA_CFG_MAX_REQUEST_SIZE=2097152
- TEST_TOPIC_NAME=test
volumes:
- ./kafka_data:/bitnami/kafka
healthcheck:
test: ["CMD", "kafka-topics.sh", "--bootstrap-server=localhost:9092", "--list"]
start_period: 15s
interval: 10s
# This service needs for setup kafka topics
setup-kafka:
image: *kafka-image
working_dir: /opt/bitnami/kafka/bin
entrypoint: /bin/bash
depends_on:
kafka:
condition: service_healthy
command: |
kafka-topics.sh --create --if-not-exists --topic test --replication-factor=1 --partitions=1 --bootstrap-server kafka:9092
I thought problem was in docker-compose file, but I also tried to run an instance from clear bitnami image. E.g
kafka_1:
image: 'bitnami/kafka:latest'
environment:
- KAFKA_CFG_NODE_ID=0
- KAFKA_CFG_PROCESS_ROLES=controller,broker
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
- KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093
- KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
But problem did not disappear.
Add the following env var to the environment section: KAFKA_CFG_MESSAGE_MAX_BYTES=20000000