[SOLVED] How to use AWS_MSK_IAM sasl mechanism with a kafka producer jar?

How to use AWS_MSK_IAM sasl mechanism with a kafka producer jar?

I have a fat jar called producer which produces messages.I want to use it to produce messages to a MSK serverless cluster. The jar takes the following arguments-

-topic --num-records  --record-size  --throughput --producer.config /configLocation/

As my MSK serverless cluster uses IAM based authentication, i have provided the following settings in my producer.config-

bootstrap.servers=boot-url
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

The way this jar usually works is by providing a username and password with the sasl.jaas.config property.

However, with MSK serverless we have to use the IAM role attached to our EC2 instance.

When my executing the current jar by using

java - jar producer.jar -topic --num-records  --record-size  --throughput --producer.config /configLocation/

I get the exception

Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.

I don't understand how to make the producer jar find the class present in the external jar aws-msk-iam-auth-1.1.1-all.jar.

Any help would be much appreciated, Thanks.

Solution

When using strimzi operator I build custom operator image:

FROM quay.io/strimzi/operator:0.42.0 AS base
COPY aws-msk-iam-auth-1.1.1-all.jar /opt/strimzi/lib/
ENV JAVA_CLASSPATH ${JAVA_CLASSPATH}:/opt/strimzi/lib/aws-msk-iam-auth-1.1.1-all.jar

and the error is gone