apache-kafkaconfluent-platformkafka-restconfluent-cloud

Can't produce to Confluent Kafka topic through Kafka REST API


I have a trouble producing to Kafka topic through Rest API proxy.

I have a running confluent kafka cluster in which I'd like to create a topic and produce a message to it using REST API.

For that purpose I have followed the documentation and created API key and secret.

I manage to create topic:

curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>"  \
-H "Content-Type: application/json" \
-d "{\"topic_name\":\"test1\",\"partitions_count\":6,\"configs\":[]}" \
https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics" | jq 

-------------------
returns:
{
  "kind": "KafkaTopic",
  "metadata": {
    "self": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1",
    "resource_name": "crn:///kafka=lkc-1zoqz/topic=test1"
  },
  "cluster_id": "lkc-1zoqz",
  "topic_name": "test1",
  "is_internal": false,
  "replication_factor": 0,
  "partitions_count": 0,
  "partitions": {
    "related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/partitions"
  },
  "configs": {
    "related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/configs"
  },
  "partition_reassignments": {
    "related": "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud/kafka/v3/clusters/lkc-1zoqz/topics/test1/partitions/-/reassignment"
  },
  "authorized_operations": []
}

MY PROBLEM: I can't produce to that topic (can't produce to ANY topic through Kafka Rest API) :

curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>"  \
-H "Content-Type: application/json" \
--data "{"records":[{"value":"S2Fma2E="}]}" \
"https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics/test1"  
    

-----------------
returns:
    {"error_code":405,"message":"HTTP 405 Method Not Allowed"}

ALSO TRIED LIKE THIS:

curl -X POST -H "Authorization: Basic <BASE64_ENCODED_AUTH_KEY_SECRET>"  \
                   -H "Content-Type: application/json" \
                    -d  "{ \"key\": {\"value\": \"S2Fma2E=\"} }" \
                   "https://pkc-xmzwx.europe-central2.gcp.confluent.cloud:443/kafka/v3/clusters/lkc-1zoqz/topics/test1/records"  

----------------
Returns the same:

    {"error_code":405,"message":"HTTP 405 Method Not Allowed"}

IDK if this has something to do with ACL management? Looking into that right now....


Solution

  • Are you using Confluent Platform (hosted) or Confluent Cloud API?

    If the latter, this API is not supported, here's the documentation on what's supported by Cloud API: https://docs.confluent.io/cloud/current/api.html