I'm using kafka for a project and specifically I'm using Confluent Cloud's managed kafka. I need to update my broker configuration (specifically the field offsets.retention.minutes), but I'm at a bit of a loss at how I'm supposed to do this. Their online console provides no way to do so that I can see, I've tried using their proprietary 'confluent' CLI which claims to be able to accomplish what I want (via: https://docs.confluent.io/confluent-cli/current/command-reference/kafka/broker/confluent_kafka_broker_update.html#confluent-kafka-broker-update), but I have to pass a URL to that tool to execute that particular command (other commands work fine, but not the broker ones) and the tool fails when I pass it the default REST URL provided in the confluent cloud UI.
For completeness though, the commands I've tried to run are in the following forms:
confluent kafka broker update --all --url https://<CLUSTER_ID>.us-west1.gcp.confluent.cloud:443 --config offsets.retention.minutes=262800
which yields: "Error: unable to establish Kafka REST connection: https://<CLUSTER_ID>.us-west1.gcp.confluent.cloud:44/v3: dial tcp 35.247.31.53:44: i/o timeout" (it appears to be doing something DEEPLY weird with the port number as it has DROPPED the 3.....
confluent kafka broker update --all --url https://<CLUSTER_ID>.us-west1.gcp.confluent.cloud --config offsets.retention.minutes=262800
which yields: "Error: REST request failed: Not Found (0)"
confluent kafka broker update --all --url https://<CLUSTER_ID>.us-west1.gcp.confluent.cloud:443/kafka --config offsets.retention.minutes=262800
which yields: "Error: REST request failed: HTTP 404 Not Found (404)" (at least it's a more normal error).
I see references online about logging into the brokers and manually changing the server.properties file to include my configuration and manually restarting the server, which I'm totally fine with, but I cannot see any concrete machine that I should be able to login to to accomplish this.
So, how can I set a broker configuration value in Confluent Cloud's managed kafka offering? I am a paid customer btw, if that matters for some reason.
The answer to my question is apparently, that I cannot change this setting on the confluent cloud. It is possible to change cluster settings if you pay for a dedicated cluster, but if you do that then there is only a tiny subset of broker configuration fields that can be changed, and 'offsets.retention.minutes' is not one of them.
From: https://docs.confluent.io/platform/current/installation/configuration/topic-configs.html#compaction
You cannot edit cluster settings on Confluent Cloud on Basic or Standard clusters, but you can edit certain topic configurations after a topic has been created. For a list of editable topic settings, see Custom topic settings for all cluster types.
You can change some configuration settings on Dedicated clusters using the Kafka CLI or REST API. See Change cluster settings for Dedicated clusters.
Kudos to a very helpful confluent support rep that answered a piece of documentation feedback with some helpful links that led me to this information.