apache-kafkastrimzi

Strimzi Kafka authentication/authorization


i'm trying to create a user resource through user operator which can be used for clients to connect to kafka. i'm also implementing authorization as well.

following is my kafka-resource file.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.2.1
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
    authorization:
      type: simple
    resources:
      requests:
        memory: 1Gi
        cpu: 0.4
      limits:
        memory: 1Gi
        cpu: 0.5
    template:
      pod:
        tolerations:
          - key: "kubernetes.azure.com/scalesetpriority"
            operator: "Equal"
            value: "spot"
            effect: "NoSchedule"
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
      inter.broker.protocol.version: "3.1"
    storage:
      type: ephemeral
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
  zookeeper:
    replicas: 1
    storage:
      type: ephemeral
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: zookeeper-metrics-config.yml
  kafkaExporter:
    topicRegex: ".*"
    groupRegex: ".*"

and below is my user-resource file

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: kafka-user1
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: "scram-sha-512"
  authorization:
    type: "simple"
    acls:
      - resource:
          type: "topic"
          name: "sit-*"
          patternType: "prefix"
        operation: "Read"
      - resource:
          type: "topic"
          name: "sit-*"
          patternType: "prefix"
        operation: "Write"
      - resource:
          type: "topic"
          name: "sit-*"
          patternType: "prefix"
        operation: "Create"
      - resource:
          type: "topic"
          name: "sit-*"
          patternType: "prefix"
        operation: "Describe"

problem is

  1. i do see a user-resource being created but it never goes into READY state

$ kubectl get kafkauser.kafka.strimzi.io -n kafka

NAME CLUSTER AUTHENTICATION AUTHORIZATION READY

kafka-user1 my-cluster scram-sha-512 simple

  1. i don't see any secrets being created

$ kubectl get secret -n kafka NAME TYPE DATA AGE kafka-user-auth Opaque 2 54d my-cluster-clients-ca Opaque 1 5m55s my-cluster-clients-ca-cert Opaque 3 5m55s my-cluster-cluster-ca Opaque 1 5m55s my-cluster-cluster-ca-cert Opaque 3 5m55s my-cluster-cluster-operator-certs Opaque 4 5m55s my-cluster-kafka-brokers Opaque 4 5m32s my-cluster-kafka-exporter-certs Opaque 4 4m59s my-cluster-zookeeper-nodes Opaque 4 5m54s sh.helm.release.v1.strimzi.v1 helm.sh/release.v1 1 61d

NOTE: please ignore kafka-user-auth secret, it's a different one.

here's a kubectl describe on the user-resource

Name:         kafka-user1
Namespace:    kafka
Labels:       strimzi.io/cluster=my-cluster
Annotations:  <none>
API Version:  kafka.strimzi.io/v1beta2
Kind:         KafkaUser
Metadata:
  Creation Timestamp:  2023-05-16T15:50:56Z
  Generation:          1
  Managed Fields:
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
        f:labels:
          .:
          f:strimzi.io/cluster:
      f:spec:
        .:
        f:authentication:
          .:
          f:type:
        f:authorization:
          .:
          f:acls:
          f:type:
    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2023-05-16T15:50:56Z
  Resource Version:  26514057
  UID:               6131c71a-7fa9-4de5-9a5e-59970f346faa
Spec:
  Authentication:
    Type:  scram-sha-512
  Authorization:
    Acls:
      Operation:  Read
      Resource:
        Name:          sit-*
        Pattern Type:  prefix
        Type:          topic
      Operation:       Write
      Resource:
        Name:          sit-*
        Pattern Type:  prefix
        Type:          topic
      Operation:       Create
      Resource:
        Name:          sit-*
        Pattern Type:  prefix
        Type:          topic
      Operation:       Describe
      Resource:
        Name:          sit-*
        Pattern Type:  prefix
        Type:          topic
    Type:              simple
Events:                <none>

Solution

  • Your Kafka resource does not enable the User Operator -> without that there is nobody to handle the KafkaUser resource, create the user and password, and make it ready. So you have to enable it. You can check the examples https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/kafka/kafka-persistent.yaml#L40 or the docs: https://strimzi.io/docs/operators/latest/full/deploying.html#deploying-the-topic-operator-using-the-cluster-operator-str