spring-batchspring-batch-tasklet

Any support in Spring Batch for reading multiple files under zip


I am looking for loading people records from multiple files based on location. Is there any easy support by Spring batch to load multiple files named location weise? Easy Country_people.zip -> Location1 (folder1) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

 -> Location2 (folder2) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

 -> Location3 (folder3) containing 3 text files(people_education.txt, people_address.txt, people_income.txt)

Solution

  • You can try using Partitioner to get the data from multiple files

    https://docs.spring.io/spring-batch/docs/current/reference/html/scalability.html#partitioning

    Or https://docs.spring.io/spring-framework/docs/4.0.0.RELEASE/javadoc-api/org/springframework/core/io/support/PathMatchingResourcePatternResolver.html

    https://docs.spring.io/spring-batch/docs/current/api/org/springframework/batch/item/file/MultiResourceItemReader.html

    public class CustomMultiResourcePartitioner implements Partitioner {
     
        @Override
        public Map<String, ExecutionContext> partition(int gridSize) {
            Map<String, ExecutionContext> map = new HashMap<>(gridSize);
            int i = 0, k = 1;
            for (Resource resource : resources) {
                ExecutionContext context = new ExecutionContext();
                Assert.state(resource.exists(), "Resource does not exist: " 
                  + resource);
                context.putString(keyName, resource.getFilename());
                context.putString("opFileName", "output"+k+++".xml");
                map.put(PARTITION_KEY + i, context);
                i++;
            }
            return map;
        }
    }