I am getting the error when attempting to setup Debezium in kubernetes (minikube for now) using Strimzi. What is wrong with my manifest? I may be confusing how to use the debezium connect image. I am new to debezium and trying to setup a local test in minikube. Thanks.
kubectl create ns kafka
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka
kubectl apply -f https://strimzi.io/examples/latest/kafka/kraft/kafka-single-node.yaml -n kafka
Error:
kafka -n kafka logs my-connect-cluster-connect-0
Using BOOTSTRAP_SERVERS=10.99.215.168:9092
Plugins are loaded from /kafka/connect
/docker-entrypoint.sh: line 330: /opt/kafka/kafka_connect_run.sh: No such file or directory
Kubernetes manifest:
apiVersion: v1
kind: Service
metadata:
name: postgres
namespace: kafka
spec:
ports:
- port: 5432
targetPort: 5432
selector:
app: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
namespace: kafka
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: quay.io/debezium/example-postgres:2.7
ports:
- containerPort: 5432
env:
- name: POSTGRES_USER
value: postgres
- name: POSTGRES_PASSWORD
value: postgres
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
name: my-connect-cluster
namespace: kafka
spec:
version: 3.0.0
replicas: 1
bootstrapServers: my-cluster-kafka-bootstrap:9092
image: quay.io/debezium/connect:latest
resources:
requests:
memory: 2Gi
cpu: 1
limits:
memory: 4Gi
cpu: 2
config:
group.id: connect-cluster
offset.storage.topic: connect-offsets
config.storage.topic: connect-configs
status.storage.topic: connect-status
key.converter: org.apache.kafka.connect.json.JsonConverter
value.converter: org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable: false
value.converter.schemas.enable: false
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: inventory-connector
namespace: kafka
labels:
strimzi.io/cluster: my-connect-cluster
spec:
class: io.debezium.connector.postgresql.PostgresConnector
tasksMax: 1
config:
bootstrap.servers: "my-cluster-kafka-bootstrap:9092"
database.hostname: postgres
database.port: "5432"
database.user: postgres
database.password: postgres
database.dbname: postgres
database.server.name: postgres
slot.name: debezium_slot
publication.name: debezium_pub
plugin.name: pgoutput
topic.prefix: postgres
snapshot.mode: initial
The container image you use has to be based on the Strimzi Kafka image for the corresponding Strimzi (and Kafka) version. I'm not sure that is the case with quay.io/debezium/connect:latest
as the error suggests.
You can check the Strimzi Docs that contain a guides for how to add connector plugins to the Strimzi image and you have to follow that.