Auth issue with Spring Kafka and Azure Event Hub

I have Spring Boot application that works fine when I have the Kafka broker running locally in a container.

But when I try to make use of Azure Eventhubs I get the following errors:

java.lang.RuntimeException: non-nullable field authBytes was serialized as null
    at org.apache.kafka.common.message.SaslAuthenticateResponseData.read(SaslAuthenticateResponseData.java:137) ~[kafka-clients-3.6.1.jar:na]
    at org.apache.kafka.common.message.SaslAuthenticateResponseData.<init>(SaslAuthenticateResponseData.java:86) ~[kafka-clients-3.6.1.jar:na]

Dependencies and versions:

• Spring Boot 3.2.4
• Spring Kafka 3.1.3

This is how the auth-specific Kafka part of my application-azure.yaml looks like:

spring:
  kafka:
    bootstrap-servers: {namespace}.servicebus.windows.net:9093
    properties:
      security:
        protocol: SASL_SSL
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required 
            username="{namespace}.servicebus.windows.net:9093" 
            password="Endpoint=sb://{namespace}.servicebus.windows.net/;SharedAccessKeyName=KafkaAccess;SharedAccessKey={accesskey}";
      ssl:
        endpoint.identification.algorithm: https

Any ideas are highly appreciated.

scale set azure agents pipelines are too slow in azure devops

I encountered the same issue when connecting a Spring Boot application to Azure Event Hubs using the Kafka protocol:

java.lang.RuntimeException: non-nullable field authBytes was serialized as null
    at org.apache.kafka.common.message.SaslAuthenticateResponseData.read(SaslAuthenticateResponseData.java:137)

This error occurred due to incorrect JAAS configuration values. Initially, I used the Event Hubs namespace and SAS key directly as the username, which is not supported.

Below is the corrected application-azure.yaml that resolved the issue:

spring:
  kafka:
    bootstrap-servers: {namespace}.servicebus.windows.net:9093
    properties:
      security:
        protocol: SASL_SSL
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="$ConnectionString"
            password="Endpoint=sb://{namespace}.servicebus.windows.net/;SharedAccessKeyName=KafkaAccess;SharedAccessKey={accesskey}";
      ssl:
        endpoint.identification.algorithm: https

Sample Configuration Details:

• username must be exactly $ConnectionString

• password must be the full Azure Event Hubs connection string (SAS policy with send or listen access)

After applying this configuration, the Spring Kafka client connected successfully and started producing events without errors.

As per the Microsoft documentation on Kafka with Azure Event Hubs, this is the expected configuration pattern when using SASL/PLAIN.