apache-kafkaapache-zookeeperdisksolid-state-drive

Does Zookeeper require SSD disks for Apache Kafka Clusters?

we want to install kafka cluster and 3 zookeeper servers

kafka should use the zookeeper servers in order to save the metadata on the zookeeper servers

ZK Data and Log files should be on disks, which have least contention from other I/O activities. Ideally the ZK data and ZK transaction log files should be on different disks, so that they don't contend for the IO resource.

Note that, it isn't enough to just have partitions but they have to be different disks to ensure performance.

So dose zookeeper server must use SSD disks ?

if yes what are the minimum requirements for zoo disks as IO ,etc.

enter image description here

Solution

  • Confluent recommends the following configuration when running Zookeeper in Production environments:

    Disks
    Disk performance is vital to maintaining a healthy ZooKeeper cluster. Solid state drives (SSD) are highly recommended as ZooKeeper must have low latency disk writes in order to perform optimally. Each request to ZooKeeper must be committed to to disk on each server in the quorum before the result is available for read. A dedicated SSD of at least 64 GB in size on each ZooKeeper server is recommended for a production deployment. You can use autopurge.purgeInterval and autopurge.snapRetainCount to automatically cleanup ZooKeeper data and lower maintenance overhead.