I am working on a SpringBatch project that needs to read from an XML and write data into a MongoDB.
I was happily surprised to see that SpringBatch has a brand new job repository support for MongoDB, so I decided to try it.
Unfortunaly I can't figure out how to configure Spring to have the batch metadata automatically initialized with the first start of the batch. I must specify that I don't want an embedded datasource for this batch but an external one to handle the retry process on failure for a potentially long process.
I have been initializing my project with this configuration:
@Bean
public JobLauncher jobLauncher(JobRepository jobRepository) {
TaskExecutorJobLauncher jobLauncher = new TaskExecutorJobLauncher();
jobLauncher.setJobRepository(jobRepository);
return jobLauncher;
}
@Bean
public MongoDatabaseFactory mongoDatabaseFactory(){
return new SimpleMongoClientDatabaseFactory("<myMongoDBconnectionString>");
}
@Bean
public MongoTemplate mongoTemplate(MongoDatabaseFactory databaseFactory) {
return new MongoTemplate(databaseFactory);
}
@Bean
public MongoTransactionManager transactionManager(MongoDatabaseFactory databaseFactory) {
MongoTransactionManager mongoTransactionManager = new MongoTransactionManager();
mongoTransactionManager.setDatabaseFactory(databaseFactory);
mongoTransactionManager.afterPropertiesSet();
return mongoTransactionManager;
}
@Bean
public JobRepository jobRepository(MongoTemplate mongoTemplate, MongoTransactionManager transactionManager) throws Exception {
MongoJobRepositoryFactoryBean jobRepositoryFactoryBean = new MongoJobRepositoryFactoryBean();
jobRepositoryFactoryBean.setMongoOperations(mongoTemplate);
jobRepositoryFactoryBean.setTransactionManager(transactionManager);
jobRepositoryFactoryBean.afterPropertiesSet();
return jobRepositoryFactoryBean.getObject();
}
But the when I execute my SpringBatchApplication, I got this exception :
java.lang.NullPointerException: Cannot invoke "org.bson.Document.getLong(Object)" because the return value of "com.mongodb.client.MongoCollection.findOneAndUpdate(org.bson.conversions.Bson, org.bson.conversions.Bson, com.mongodb.client.model.FindOneAndUpdateOptions)" is null
at org.springframework.batch.core.repository.dao.MongoSequenceIncrementer.lambda$nextLongValue$0(MongoSequenceIncrementer.java:51) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:603) \~\[spring-data-mongodb-4.4.1.jar:4.4.1\]
at org.springframework.batch.core.repository.dao.MongoSequenceIncrementer.nextLongValue(MongoSequenceIncrementer.java:47) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at org.springframework.batch.core.repository.dao.MongoJobInstanceDao.createJobInstance(MongoJobInstanceDao.java:80) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:168) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) \~\[na:na\]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) \~\[na:na\]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) \~\[na:na\]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) \~\[na:na\]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359) \~\[spring-aop-6.2.1.jar:6.2.1\]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) \~\[spring-aop-6.2.1.jar:6.2.1\]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) \~\[spring-aop-6.2.1.jar:6.2.1\]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380) \~\[spring-tx-6.2.1.jar:6.2.1\]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) \~\[spring-tx-6.2.1.jar:6.2.1\]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) \~\[spring-aop-6.2.1.jar:6.2.1\]
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean.lambda$getObject$0(AbstractJobRepositoryFactoryBean.java:204) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) \~\[spring-aop-6.2.1.jar:6.2.1\]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:223) \~\[spring-aop-6.2.1.jar:6.2.1\]
at jdk.proxy2/jdk.proxy2.$Proxy41.createJobExecution(Unknown Source) \~\[na:na\]
at org.springframework.batch.core.launch.support.TaskExecutorJobLauncher.run(TaskExecutorJobLauncher.java:143) \~\[spring-batch-core-5.2.1.jar:5.2.1\]
at com.docoon.update_annuaire_replica.SpringBatchApplication.run(SpringBatchApplication.java:39) \~\[classes/:na\]
It seems like it is looking for a particular sequence in the metadata collection, but cannot find it beacause the schema haven't been intialized yet. I know that with an SQL
configuration this property can be set :
spring.main.batch.jdbc.initalize-schema: true
Is there any equivalence for this for a MongoDB config that I am missing ?
Spring Boot does not support initializing mongodb with batch meta-data because it is not a standard sql file (it is rather a js file). So you need to initialize the database manually before starting your first job.
You can find a complete docker-based example here: https://github.com/spring-projects/spring-batch/blob/main/spring-batch-core/src/test/java/org/springframework/batch/core/repository/support/MongoDBJobRepositoryIntegrationTests.java
Please take a look at how the database is initialized with a mongo template.