elasticsearchapache-kafka-connect

Bad Request when connecting to Elasticsearch from ElasticsearchSinkConnector


I have Elasticsearch running locally and Confluent Platform hosted in Docker.

Here is how I created a sink connector:

{
 "name": "elasticsearch_product_listings_sink",
 "config": {
   "connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
   "connection.url": "http://host.docker.internal:9200",
   "tasks.max": "1",
   "topics": "mysql-products",
   "name": "elasticsearch_product_listings_sink",
   "type.name": "listing",
   "key.ignore": "true"
 }
}

Request to Elasticsearch http://localhost:9200 returns

{
    "name": "DESKTOP-82EPB3S",
    "cluster_name": "elasticsearch",
    "cluster_uuid": "NeNyRrtXQPqUD1EN3hHpsA",
    "version": {
        "number": "9.0.2",
        "build_flavor": "default",
        "build_type": "zip",
        "build_hash": "0a58bc1dc7a4ae5412db66624aab968370bd44ce",
        "build_date": "2025-05-28T10:06:37.834829258Z",
        "build_snapshot": false,
        "lucene_version": "10.1.0",
        "minimum_wire_compatibility_version": "8.18.0",
        "minimum_index_compatibility_version": "8.0.0"
    },
    "tagline": "You Know, for Search"
}

But Docker logs show me this error:

ERROR WorkerSinkTask{id=elasticsearch_product_listings_sink1-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)

....

Caused by: org.elasticsearch.client.ResponseException: method [HEAD], host [http://host.docker.internal:9200], URI [/mysql-products], status line [HTTP/1.1 400 Bad Request]

What am I missing?


Solution

  • Looks like 9.0.2 version of Elasticsearch is not compatible with ElasticsearchSinkConnector.

    Once I switched to 8.x version (8.12.0) everything started to work.