i'm tyring to enable authentication on strimzi kafka. below is my yml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
version: 3.2.1
replicas: 1
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
authentication:
type: scram-sha-512
usernameSecretRef:
name: kafka-user-auth
key: kafka-user
passwordSecretRef:
name: kafka-user-auth
key: kafka-pass
resources:
requests:
memory: 1Gi
cpu: 0.4
limits:
memory: 1Gi
cpu: 0.5
template:
pod:
tolerations:
- key: kubernetes.azure.com/scalesetpriority
operator: Equal
value: spot
effect: NoSchedule
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
default.replication.factor: 1
min.insync.replicas: 1
inter.broker.protocol.version: '3.1'
storage:
type: persistent-claim
size: 10Gi
deleteClaim: true
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: kafka-metrics-config.yml
zookeeper:
replicas: 1
storage:
type: persistent-claim
size: 2Gi
deleteClaim: true
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: zookeeper-metrics-config.yml
kafkaExporter:
topicRegex: .*
groupRegex: .*
but when applying this yml, i get below error
error: error validating "strimzi-kafka-with-auth.yml": error validating data: [ValidationError(Kafka.spec.kafka.listeners[1].authentication): unknown field "passwordSecretRef" in io.strimzi.kafka.v1beta2.Kafka.spec.kafka.listeners.authentication, ValidationError(Kafka.spec.kafka.listeners[1].authentication): unknown field "usernameSecretRef" in io.strimzi.kafka.v1beta2.Kafka.spec.kafka.listeners.authentication]; if you choose to ignore these errors, turn validation off with --validate=false
i tried different approaches, e.g.
authentication:
type: scram-sha-512
username: myuser
passwordSecret:
secretName: myuser-secret
passwordKey: password
and this
authentication:
type: sasl_plaintext
usernameSecret:
secretKeyRef:
name: myuser-secret
key: myuser-username
passwordSecret:
secretKeyRef:
name: myuser-secret
key: myuser-password
and this
authentication:
type: scram-sha-512
username: my-kafka-user
password: my-kafka-password
but none of the above approaches seem to work, i get errors like it's not recognizing either username
or password
fields or not recognizing usernameSecret
and passwordSecret
fields
UPDATE
i am now creating kafka using below yml which works fine
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
version: 3.2.1
replicas: 1
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
authentication:
type: scram-sha-512
resources:
requests:
memory: 1Gi
cpu: 0.4
limits:
memory: 1Gi
cpu: 0.5
template:
pod:
tolerations:
- key: "kubernetes.azure.com/scalesetpriority"
operator: "Equal"
value: "spot"
effect: "NoSchedule"
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
default.replication.factor: 1
min.insync.replicas: 1
inter.broker.protocol.version: "3.1"
storage:
type: ephemeral
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: kafka-metrics-config.yml
zookeeper:
replicas: 1
storage:
type: ephemeral
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: zookeeper-metrics-config.yml
kafkaExporter:
topicRegex: ".*"
groupRegex: ".*"
entityOperator:
topicOperator: {}
userOperator: {}
and a user using user operator
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
name: my-user
labels:
strimzi.io/cluster: my-cluster
spec:
authentication:
type: scram-sha-512
now when i try to connect kafka client, it requires ssl certificates. however i want kafka broker to authenticate the client using username/pass and transport data over tls without certificate verification. is it possible? i don't want any certificates to be placed on client side
The Kafka
custom resource does not configure any passwords or users. It only configures the authentication type. The actual users and their credentials are separate from that. You can create users for example using the KafkaUser
resources through the User operator. Or you can do it directly in Kafka using Kafka Admin APIs if you don't want to use the User Operator. Or with some authentication methods such as OAuth, the users live somewhere else completel.
You can check out the examples: https://github.com/strimzi/strimzi-kafka-operator/tree/main/examples/security and / or read the docs which have some examples as well (e.g. https://strimzi.io/docs/operators/latest/full/deploying.html#assembly-securing-kafka-str). You can also check the API reference which shows which fields actually exist and can be used.