First of all, I've seen this thread but it's unrelated and having different issue.
I have the following settings fragment in my Kafka properties file:
ssl.keystore.type=PEM
ssl.keystore.key=/path/to/private.key
ssl.keystore.certificate.chain=/path/to/certificate.pem
ssl.truststore.type=PEM
ssl.truststore.certificates=/path/to/ca.pem
ssl.endpoint.identification.algorithm=
Note that ssl.endpoint.identification.algorithm is used because single-server certificate is used for each server in a cluster, therefore I have to bypass SSL hostname verification this way.
When starting Kafka, I am getting the following:
org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:107)
at kafka.network.Processor.<init>(SocketServer.scala:853)
at kafka.network.SocketServer.newProcessor(SocketServer.scala:442)
at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors$1(SocketServer.scala:299)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:297)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1(SocketServer.scala:262)
at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1$adapted(SocketServer.scala:259)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:259)
at kafka.network.SocketServer.startup(SocketServer.scala:131)
at kafka.server.KafkaServer.startup(KafkaServer.scala:285)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: No matching PRIVATE KEY entries in PEM file
Thing is - private.key, certificate.pem and ca.pem are valid files and working with other applications/client libraries. Out of them I used to create keystore/truststore and it works fine. When using keystore/truststore, I've also used these 3 files to connect to Kafka from Python and it worked just fine. I confirm these files are valid and have no issues with other applications.
Also note that private key is PKCS#8 type, which is expected by Kafka:
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
What am I doing wrong?
You need to specify PEM file contents when using PEM store types and ssl.keystore.key and ssl.keystore.certificate.chain properties:
security.protocol=SSL
ssl.keystore.type=PEM
ssl.keystore.key=-----BEGIN PRIVATE KEY----- \
................................................................ \
.........................................= \
-----END PRIVATE KEY-----
ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE----- \
................................................................ \
-----END CERTIFICATE-----