I want to load multiple CSV files into mysql database at single table using Spring Batch. The path of the files are derived from the following method.
public List<String> getFilePath() {
String inputPath = "E:\\input";
List<String> inputCSVPaths = new ArrayList<String>();
Map<String, List<String>> inputInfo = new HashMap<String, List<String>>();
File inputFolder = new File(inputPath);
File[] inputFiles = inputFolder.listFiles();
for (File file : inputFiles) {
inputCSVPaths.add(file.getAbsolutePath());
}
inputInfo.put("Introduction", inputCSVPaths);
List<String> inputFile = inputInfo.get("Introduction");
System.out.println("Input File :"+inputFile);
return inputFile;
}
There are total 3 CSV files. But it reads only onle file and inserts data of only that CSV file. Is there wrong in getting resources.
@Autowired
private FilePathDemo filePathDemo;
@Bean
public MultiResourceItemReader<Introduction> multiResourceItemReader() throws IOException {
MultiResourceItemReader<Introduction> multiReader = new MultiResourceItemReader<Introduction>();
ResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
Resource[] resources;
String filePath = "file:";
List<String> path = filePathDemo.getFilePath();
for (String introPath : path) {
System.out.println("File Path of the Introduction CSV :" + introPath);
resources = patternResolver.getResources(filePath + introPath);
multiReader.setResources(resources);
}
FlatFileItemReader<Introduction> flatReader = new FlatFileItemReader<Introduction>();
multiReader.setDelegate(flatReader);
flatReader.setLinesToSkip(1);
flatReader.setLineMapper(new DefaultLineMapper<Introduction>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(new String[] { "id", "name", "age", "phoneNo"});
}
});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Introduction>() {
{
setTargetType(Introduction.class);
}
});
}
});
flatReader.close();
multiReader.close();
return multiReader;
}
There are two issues with your configuration:
resources
array with a single file in the for loop. Hence, the MultiResourceItemReader
will be configured with only one file.close
method on the MultiResourceItemReader
and the delegate FlatFileItemReader
but you should not. Spring Batch will call those methods when the step is complete.You can find an example of how to configure the MultiResourceItemReader
here: https://docs.spring.io/spring-batch/4.0.x/reference/html/readersAndWriters.html#multiFileInput