I have been try to run Zookeeper and Kafka on Docker container.
I got a lot of errors [error occurred during error reporting , id 0xb]
and [Too many errors, abort]
in my terminal. And then library initialization failed - unable to allocate file descriptor table - out of memory
.
I use the following docker-compose.yml file
version: "3.8"
services:
zookeeper:
image: confluentinc/cp-zookeeper:5.5.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: confluentinc/cp-kafka:5.5.4
# If you want to expose these ports outside your dev PC,
# remove the "127.0.0.1:" prefix
ports:
- 127.0.0.1:9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
This problem is caused by having too many file handles available to the program. It increases its soft limit on files to the maximum available, then attempts to allocate memory for each of those file handles. For some systems, with high limits, this causes insufficent memory.
It can be fixed by overriding the command arguments of ExecStart
in docker.service
sudo systemctl edit docker
and then enter
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --default-ulimit nofile=65536:65536 -H fd://
Save and run sudo systemctl daemon-reload
and sudo systemctl restart docker
.