javaspring-bootspring-batchspring-kafkaclassnotfoundexception

Spring Kafka: ClassNotFoundException: com.google.common.cache.CacheLoader

I write you this message because I have an error when trying to execute a Spring batch job that reads messages from a Kafka topic.

This job is triggered after calling an URL in my web app. The final error message says:

Caused by: java.lang.ClassNotFoundException: com.google.common.cache.CacheLoader

So maybe I need to include other dependencies in my pom. I have the same error both locally on my laptop or on the acceptance environement (On Openshift) where the app is deployed.

Also when the same batch is executed as a CronJob on Openshift (in another module), it works well.

Could you help find out which dependency I should add ?

Here is some relevant chunks of code that shows how I run my code: Controller

@PostMapping (value = "/launch-consumption", produces = MediaType.APPLICATION_JSON_VALUE)
public @ResponseBody ResponseEntity<String> consumeTopic() throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
    log.info("Received a request to launch the consumer job");

    // If batch already running, send HTTP 202
    if(service.isBatchAlreadyRunning()) {
        return ResponseEntity.status(HttpStatus.ACCEPTED).body(BATCH_ALREADY_RUNNING);
    }
    asyncLaunchBatch();
    return ResponseEntity.status(HttpStatus.OK).body(BATCH_JUST_LAUNCHED);
}

@Async
private void asyncLaunchBatch() throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
    service.launchConsumerBatch();
}

The Service

@Service
public class Service {
    @Autowired
    private JobLauncher jobLauncher;
    @Autowired
    @Qualifier("jobConsumer")
    private Job job;
    @Autowired
    private JobExplorer jobExplorer;

    public void launchConsumerBatch() throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
        // Add launch date to parameter.
        // A job is uniquely identified by its id together with the parameters.
        // So we add the launch date as a parameter
        // So that we won't be blocked when launching a job multiple times.
        JobParametersBuilder builder = new JobParametersBuilder();
        builder.addDate("date", new Date());

        jobLauncher.run(job, builder.toJobParameters());
    }

}

The job:

@Bean
public Job jobConsumer(
        JobRepository jobRepository,
        Step stepTruncateTable,
        Step stepReadProcessWrite,
        ReceptionSuccessDecider receptionSuccessDecider,
        Step stepBackupLastDailyTrafics,
        Step stepRestoreFromBackupDailyTrafics,
        Step stepRestoreQualificationState) {
    return new JobBuilder(JOB_NAME, jobRepository)
            .listener(new JobListener())
            .incrementer(new CustomIncrementer(JOB_NAME))
            .start(stepTruncateTable)
            .next(stepReadProcessWrite)
            .next(receptionSuccessDecider).on("FAILED").to(stepRestoreFromBackupDailyTrafics)
            .next(stepRestoreQualificationState)
            .from(receptionSuccessDecider).on("COMPLETED").to(stepBackupLastDailyTrafics)
            .next(stepRestoreQualificationState)
            .on("COMPLETED").end()
            .on("FAILED").fail().end()
            .build();
}

KafkaItemReader:

public class DailyTraficToQualifyKafkaItemReader extends KafkaItemReader<String, DailyTrafficToQualify> {

    public DailyTraficToQualifyKafkaItemReader(Properties consumerProperties, String topicName, List<Integer> partitions) {
        super(consumerProperties, topicName, partitions);
        // Passing an empty map makes the reader start from the offset stored in Kafka
        // for the consumer group ID.
        // In case of a restart, offsets stored in the execution context
        // will take precedence.
        setPartitionOffsets(new HashMap<>());
        setName("DailyTraficToQualifyReader");
    }

}

Below is my pom and then the error stack trace:

Relevant content in parent pom and then module pom:

<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.1.4</version>
</parent>

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>module-name</artifactId>
<packaging>jar</packaging>
<name>module-name</name>
<description>Backend qualification</description>
<!-- déclaration de spring boot en projet parent TDC-CTR-->
<parent>
    <groupId>com.mycompany.projectname</groupId>
    <artifactId>pom-parent</artifactId>
    <version>6.0.0.18-SNAPSHOT</version>
    <relativePath>../pom-parent/pom.xml</relativePath>
</parent>

<dependencies>
    <dependency>
        <groupId>com.mycompany.projectname.library</groupId>
        <artifactId>lib-mysql</artifactId>
        <version>6.0.0.18-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>com.mycompany.projectname.library</groupId>
        <artifactId>lib-swagger</artifactId>
        <version>6.0.0.18-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>com.mycompany.projectname.library</groupId>
        <artifactId>lib-common</artifactId>
        <version>6.0.0.18-SNAPSHOT</version>
    </dependency>
    <!-- core dependencies -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-validation</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.10.1</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-avro-serializer</artifactId>
        <version>7.2.1</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.batch</groupId>
        <artifactId>spring-batch-core</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
    </dependency>
    <!-- Spring Actuator - provides /health endpoint and some other metrics -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <!-- MySQL Dependency -->
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-jpa</artifactId>
    </dependency>
    <!-- Util dependencies -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-annotations</artifactId>
        <version>2.15.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
    </dependency>
    <dependency>
        <groupId>jakarta.persistence</groupId>
        <artifactId>jakarta.persistence-api</artifactId>
    </dependency>
    <!-- tests dependencies -->
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>20.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.batch</groupId>
        <artifactId>spring-batch-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
    </dependency>
</dependencies>

<build>
    <resources>
        <resource>
            <directory>src/main/resources</directory>
            <filtering>true</filtering>
            <includes>
                <include>**/*.properties</include>
                <include>**/*.json</include>
                <include>**/*.xml</include>
            </includes>
        </resource>
    </resources>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>3.1.0</version>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>16</source>
                <target>16</target>
            </configuration>
        </plugin>
        <!-- avro maven plugin generating java classes from idl object and avro schemas -->
        <plugin>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro-maven-plugin</artifactId>
            <version>1.10.1</version>
            <executions>
                <execution>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>schema</goal>
                    </goals>
                    <configuration>
                        <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>
                        <outputDirectory>${project.basedir}/target/generated-sources/</outputDirectory>
                        <stringType>String</stringType>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

The error stack trace:

    ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step stepReadProcessWrite in job jobConsumer
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:626)
at org.springframework.batch.item.kafka.KafkaItemReader.open(KafkaItemReader.java:168)
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:124)
...
at java.base/java.lang.Thread.run(Thread.java:840)
    Caused by: java.lang.NoClassDefFoundError: com/google/common/cache/CacheLoader
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3373)
at java.base/java.lang.Class.getConstructor0(Class.java:3578)
at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2754)
at org.springframework.kafka.support.serializer.ErrorHandlingDeserializer.setupDelegate(ErrorHandlingDeserializer.java:129)
at org.springframework.kafka.support.serializer.ErrorHandlingDeserializer.configure(ErrorHandlingDeserializer.java:114)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:717)
... 154 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.google.common.cache.CacheLoader
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 161 common frames omitted
o.s.batch.core.step.AbstractStep - Step: [stepReadProcessWrite] executed in 125ms
f.l.t.u.c.batch.listener.JobListener - JobListener::afterJob() -> jobExecution: jobConsumer, FAILED
Solution

  • To resolve this issue, you need to change the scope of the Guava dependency from test to compile

        <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>20.0</version>
        <scope>compile</scope>
    </dependency>