springspring-bootspring-batchflatfilereader

How to read flat file header and body separately in Spring Batch


i'm doing a simple batch job with Spring Batch and Spring Boot.

I need to read a flat file, separate the header data (first line) from the body data (rest of lines) for individual business logic processing and then write everything into a single file.

As you can see, the header has 5 params that have to be mapped to one class, and the body has 12 which have to be mapped to a different one.

I first thought of using FlatFileItemReader and skip the header. Then use the skippedLinesCallback to handle that line, but i couldn't figure out how to do it.

I'm new to Spring Batch and Java Config. If someone can help me writing a solution for my problem i would really aprecciate it!

I leave here the input file:

01.01.2017|SUBDCOBR|12:21:23|01/12/2016|31/12/2016
01.01.2017|12345678231234|0002342434|BORGIA RUBEN|27-32548987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,01
01.01.2017|12345673201234|2342434|ALVAREZ ESTHER|27-32533987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,02
01.01.2017|12345673201234|0002342434|LOPEZ LUCRECIA|27-32553387-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,12
01.01.2017|12345672301234|0002342434|SILVA JESUS|27-32558657-9|NC|A|2062-
00010443|142,12|30/08/2017|142,12

Cheers!

EDIT 1: This would be my first attepmt . My "body" POJO is called DetalleFacturacion and my "header" POJO is CabeceraFacturacion. The reader I thought to do it with DetalleFacturacion pojo, so i can skip the header and treat it later... however i'm not sure how to assign header's data into CabeceraFacturacion.

public FlatFileItemReader<DetalleFacturacion> readerDetalleFacturacion(){

    FlatFileItemReader<DetalleFacturacion> reader = new FlatFileItemReader<>();

    reader.setLinesToSkip(1);
    reader.setResource(new ClassPathResource("/inputFiles/GLEO-MN170100-PROCESO01-SUBDFACT-000001.txt"));

    DefaultLineMapper<DetalleFacturacion> detalleLineMapper = new DefaultLineMapper<>();

    DelimitedLineTokenizer tokenizerDet = new DelimitedLineTokenizer("|");
    tokenizerDet.setNames(new String[] {"fechaEmision", "tipoDocumento", "letra", "nroComprobante",  
                                "nroCliente", "razonSocial", "cuit", "montoNetoGP", "montoNetoG3",
                                "montoExento", "impuestos", "montoTotal"});

    LineCallbackHandler skippedLineCallback = new LineCallbackHandler() {

        @Override
        public void handleLine(String line) {

            String[] headerSeparado = line.split("|");

            String printDate = headerSeparado[0];
            String reportIdentifier = headerSeparado[1];
            String tituloReporte = headerSeparado[2];
            String fechaDesde = headerSeparado[3];
            String fechaHasta = headerSeparado[4];

            CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
            cabeceraFacturacion.setPrintDate(printDate);
            cabeceraFacturacion.setReportIdentifier(reportIdentifier);
            cabeceraFacturacion.setTituloReporte(tituloReporte);
            cabeceraFacturacion.setFechaDesde(fechaDesde);
            cabeceraFacturacion.setFechaHasta(fechaHasta);

        }
    };

    reader.setSkippedLinesCallback(skippedLineCallback);

    detalleLineMapper.setLineTokenizer(tokenizerDet);
    detalleLineMapper.setFieldSetMapper(new DetalleFieldSetMapper());
    detalleLineMapper.afterPropertiesSet();
    reader.setLineMapper(detalleLineMapper);

    // Test to check if it is saving correctly data in CabeceraFacturacion
    CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
    System.out.println("Print Date:"+cabeceraFacturacion.getPrintDate());
    System.out.println("Report Identif: 
    "+cabeceraFacturacion.getReportIdentifier());

    return reader;

}


Solution

  • You are correct . You need to use skippedLinesCallback to handle skip lines.

    You need to implement LineCallbackHandler interface and add you processing in handleLine method.

    LineCallbackHandler Interface passes the raw line content of the lines in the file to be skipped. If linesToSkip is set to 2, then this interface is called twice.

    This is how you can define Reader for the same.

    Java Config - Spring Batch 4

    @Bean
        public FlatFileItemReader<POJO> myReader() {
    
            return FlatFileItemReader<pojo>().
                    .setResource(new FileSystemResource("resources/players.csv"));
                    .name("myReader")
                    .delimited()
                    .delimiter(",")
                    .names("pro1,pro2,pro3")
                    .targetType(POJO.class)
                    .skippedLinesCallback(skippedLinesCallback)             
                    .build();
    
        }