Translated by ChatGPT
I'm encountering an error while trying to run a Kafka Connect JDBC Source Connector task(connect-standalone). The error stack trace shows an IllegalAccessError when Kafka Connect attempts to instantiate the JdbcSourceTask class. The error message indicates that it is trying to access the class org.apache.kafka.common.utils.SystemTime, but the class loader cannot access it.
Here's the relevant part of the error log:
[2024-11-15 17:29:25,300] ERROR [test-connector-mysql|task-0] Failed to start task test-connector-mysql-0 (org.apache.kafka.connect.runtime.Worker:707)
org.apache.kafka.connect.errors.ConnectException: Instantiation error
at org.apache.kafka.connect.runtime.isolation.Plugins.newPlugin(Plugins.java:142)
at org.apache.kafka.connect.runtime.isolation.Plugins.newTask(Plugins.java:340)
at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)
at org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:592)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startTask(StandaloneHerder.java:504)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.createConnectorTasks(StandaloneHerder.java:488)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.createConnectorTasks(StandaloneHerder.java:482)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.updateConnectorTasks(StandaloneHerder.java:541)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.lambda$null$2(StandaloneHerder.java:255)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: org.apache.kafka.common.KafkaException: Could not instantiate class io.confluent.connect.jdbc.source.JdbcSourceTask
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:405)
at org.apache.kafka.connect.runtime.isolation.Plugins.newPlugin(Plugins.java:140)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:501)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:485)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:401)
... 15 more
Caused by: java.lang.IllegalAccessError: failed to access class org.apache.kafka.common.utils.SystemTime from class io.confluent.connect.jdbc.source.JdbcSourceTask (org.apache.kafka.common.utils.SystemTime is in unnamed module of loader 'app'; io.confluent.connect.jdbc.source.JdbcSourceTask is in unnamed module of loader org.apache.kafka.connect.runtime.isolation.PluginClassLoader @63021689)
at io.confluent.connect.jdbc.source.JdbcSourceTask.<init>(JdbcSourceTask.java:78)
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
... 18 more
connect-standalone.properties is almost default setting.
Here's the source.properties
name=connector-mysql
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:mysql://localhost:3306/{dbName}
connection.user=root
connection.password=xxx
table.whitelist=invoices
mode=incrementing
incrementing.column.name=id
topic.prefix=mysql_
Kafka: 3.9.0 (installed via Homebrew) MySQL: 8.0.35 Kafka-Connect-JDBC: 10.8.0 Mysql-Connector-j : 8.2.0 Java: 17 OS: macOS Sequoia 15.0.1 (M1pro) I've already configured the plugin.path to include the correct directories, but it seems like there's a class loader conflict between Kafka Connect's core libraries and the JDBC connector.
Has anyone encountered a similar issue or know how to resolve this? It seems like the PluginClassLoader is unable to access classes that are part of Kafka's core libraries. How can I fix this class loader issue to allow proper access between the connector and the core Kafka libraries?
I have tried Kafka Connect JDBC versions 10.4.x, 10.5.x, and 10.7.x, along with MySQL Connector/J versions 8.0.33 and 9.1.0, but the same issue persists.
Self answer!
The issue was resolved by downgrading Kafka to version 3.8.x. According to the release notes for Kafka 3.9.0, the SystemTime class was changed to a singleton, and instead of creating an instance with new SystemTime();, it was modified to obtain the instance by calling the SystemTime.getSystemTime(); method.
However, the latest version of Kafka Connect JDBC(v10.8.0) initializes the field with time = new SystemTime();, which caused the error.
I have created an issue on the Kafka Connect JDBC GitHub.
Here are the relevant links:
https://issues.apache.org/jira/browse/KAFKA-16879