restapache-kafkakafka-rest

Kafka - increase partition count of existing topic through Confluent REST Api


I need to add partitions to an existing Kafka topic. I'm aware that it is possible to use the /bin/kafka-topics.sh script to achieve this, but I would prefer to do this through the Confluent REST api.

As far as I see there is no documented endpoint in the api reference, but I wonder if someone else here was able to make this work.

Edit: As it does seem to be impossible to use the REST api here, I wonder what the best practice is for adding partitions to an existing topic in a containerized setup. E.g. if there is a custom partioning scheme that maps customer ids to specific partitions. In this case the app container would need to adjust the partition count of the kafka container.


Solution

  • This is the solution I ended up with:

    For a production setup the AdminClient is preferrable, I decided for the integrated script approach because of the chosen language (rust).