dockerapache-kafkaapache-kafka-connectconfluent-platformconfluent-control-center

Docker image for Confluent - adding Confluent Hub connectors

I wanted to slightly modify Confluent's Git repo Dockerfile to have in my Confluent Connect page mongoDB and Snowflake connections. Everything runs ok but I don't see them in the portal.

Should docker-compose.yml be modified as well?

Original code:

FROM confluentinc/cp-server-connect-base:5.5.1

ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"

ARG CONNECTOR_NAME
RUN confluent-hub install --no-prompt confluentinc/${CONNECTOR_NAME}:5.5.0

Moded code:

FROM confluentinc/cp-server-connect-base:5.5.1

ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"

ARG CONNECTOR_NAME
RUN confluent-hub install --no-prompt confluentinc/${CONNECTOR_NAME}:5.5.0 \
   && confluent-hub install --no-prompt mongodb/kafka-connect-mongodb:1.2.0 \
   && confluent-hub install --no-prompt snowflakeinc/snowflake-kafka-connector:1.4.3
Solution

  • I think you can try to do the following.

    1. Modify your Dockerfile:
    FROM confluentinc/cp-server-connect-base:5.5.1

ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"

RUN confluent-hub install --no-prompt mongodb/kafka-connect-mongodb:1.2.0 \
   && confluent-hub install --no-prompt snowflakeinc/snowflake-kafka-connector:1.4.3

    Since you need to install only mongodb and snowflake connectors.

    1. Use your custom image in docker-compose.yml:
    ...
  connect:
    # image: cnfldemos/cp-server-connect-datagen:0.3.2-5.5.0
    build: .
    hostname: connect
    container_name: connect
...