SpringBatch on AWS Batch

Can I create a spring batch and invoke using AWS Batch?. Steps are recommended if so

Based on this analysis I have to take a call to use spring batch or go for AWS Batch with java application.

Yes, you can and using spring batch (and spring boot) is a good choice for building a java app that runs in aws batch. AWS batch launches a docker container and runs a command in that container to run the spring boot/batch application.

If you're also using spring boot with spring batch, you should implement a org.springframework.boot.CommandLineRunner to launch your job. The CommandLineRunner run(String ...args) method receives the command line arguments that the AWS Batch job is configured with. The CommandLineRunner converts these command line arguments into JobParameters and then launches the job.

AWS Batch uses the exit code of the java application to determine if the job succeeded or failed. This means you need to convert the JobExecution returned from running the batch job into an exit code to tell AWS Batch if the job succeeded or failed.

An example CommandLineRunner that launches the spring batch job and converts the JobExecution into an exit code is below...

@Component
@Profile("!test")
public class BatchJobCommandLineRunner implements CommandLineRunner, ExitCodeGenerator {

    private static final Logger log = LoggerFactory.getLogger(BatchJobCommandLineRunner.class);

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job awsBatchJob;

    @Autowired
    private BatchJobProperties batchJobProperties;

    private JobExecution jobExecution;

    @Override
    public void run(String... args) throws Exception {
        JobParameters jobParameters = new DefaultJobParametersConverter().getJobParameters(batchJobProperties);
        this.jobExecution = jobLauncher.run(awsBatchJob, jobParameters);
        for (StepExecution se : jobExecution.getStepExecutions()) {
            log.info(se.getSummary());
        }
    }

    @Override
    public int getExitCode() {
        int exitCode = getExitCode(jobExecution);
        log.debug("returning exitCode {}", exitCode);
        return exitCode;
    }

    private int getExitCode(JobExecution jobExecution) {
        if (jobExecution != null) {
            switch (jobExecution.getStatus()) {
            case COMPLETED:
            case STARTED:
            case STARTING:
            case STOPPING:
            case ABANDONED:
                return 0;

            case FAILED:
                return 1;

            case STOPPED:
                return 2;

            case UNKNOWN:
                return 3;
            }
        }

        return 0;
    }

}

In the above, I'm using a BatchJobProperties class to read properties from the command line. Any command line argument in the form of -Djob.property1 -Djob.property2 ... will populate the BatchJobProperties object by using the ConfigurationProperties annotation...

@Configuration
@ConfigurationProperties(prefix = "job")
public class BatchJobProperties extends Properties {}

Since this is a console application, not a web application, you configure spring boot to not create a web application in your application.yml file. Also note that spring.batch.job.enabled is set to false because the job is launched using the CommandLineRunner...

spring:
  main:
    web-application-type: NONE
  batch:
    job:
      enabled: false

When building your spring boot app, after building the jar, you build a docker container. A sample Dockerfile for this is below...

FROM eclipse-temurin:17-jdk-alpine
RUN addgroup -S spring && adduser -S spring -G spring
RUN mkdir /app
RUN chown -R spring:spring /app
RUN chmod 755 /app
WORKDIR /app
USER spring:spring
COPY target/*.jar /app.jar
CMD ["java","-jar","/app.jar"]

In AWS Batch, when you create your job instance you specify the CMD that is used when running the docker instance and this is where you specify the command line arguments. An example AWS Job instance showing where you configure the CMD used for running the docker container is below. This is where you configure the JobParameters used by your spring batch job.