I have the following component in k8s:
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: kafka-pubsub
namespace: default
spec:
type: pubsub.kafka
version: v1
metadata:
# Kafka broker connection setting
- name: brokers
value: "kafka-0.kafka-headless.default.svc.cluster.local:9092,
kafka.default.svc.cluster.local:9092"
- name: authRequired
value: "false"
- name: saslUsername
value: ""
- name: saslPassword
value: ""
scopes:
- signalrconnectorinterface
- cardhandler
- eventtrigger
I have a number of microservices which all subscribe properly, however the ones which subscribe to two topics (and more presumably) get the following error message upon subscription in the daprd logs:
time="2023-08-04T13:41:54.24563974Z" level=info msg="initializing Dapr workflow component" app_id=cardhandler component="dapr (workflow.dapr/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
time="2023-08-04T13:41:54.263847422Z" level=info msg="app is subscribed to the following topics: [send-card receive-command-message] through pubsub=kafka-pubsub" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2
time="2023-08-04T13:41:54.267674781Z" level=info msg="placement tables updated, version: 0" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime.actor.internal.placement type=log ver=1.11.2
time="2023-08-04T13:41:55.200932919Z" level=error msg="Error consuming [receive-command-message]. Retrying...: kafka: tried to use a consumer group that was closed" app_id=cardhandler component="kafka-pubsub (pubsub.kafka/v1)" instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.contrib type=log ver=1.11.2
time="2023-08-04T13:41:58.812244394Z" level=info msg="dapr initialized. Status: Running. Init Elapsed 4843ms" app_id=cardhandler instance=cardhandler-deployment-ff8d86465-xt85c scope=dapr.runtime type=log ver=1.11.2
It notices there are two subscriptions, tries to allocate them then fails. Any advice?
As per OneCricketeer, there is actually no issue here, this is intended behaviour. For some reason this just started happening when I was using a new helm install of Apache Kafka.
Ended up being a codebase issue so reset and all fine :)