javacsvapache-camelspring-camel

Camel 4.5.0 or greater breaks csvdataformat


After updating to 4.5.0 this route has been failing where data just do not get past the dotry, it split the file as one line per split and then hangs. Tried all version till 4.11.0 - manually tried to unmashal the data in processor but throwing npe as csvDataFormat.unmarshaller is null

currently it is working on version 4.4.0.

CsvDataFormat csvDataFormat = new CsvDataFormat().setUseMaps(true);

from("file:" + rootDir + "/test/")
                .log(LoggingLevel.INFO,"Start processing ${file:name}")
                .unmarshal().pgp(pgpFileName,pgpUserId,pgpPassword)
                .process(exchange -> { /* just to get csv header */
                    InputStream inputStream = exchange.getIn().getBody(InputStream.class);
                    try(BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream))){
                        String header = bufferedReader.readLine();
                        exchange.getIn().setHeader("CSV_HEADER",header);
                        csvDataFormat.setHeader(header.split(",")); //<- this does not work, so had to add in body below!
                        System.out.println("csvHeader is : " + header);// + " ? " + Arrays.asList(csvDataFormat.getHeader()));
                    }
                })
                .split(body().tokenize("\n")).shareUnitOfWork()
                .parallelProcessing().streaming()
                .setBody(exchange -> exchange.getIn().getHeader("CSV_HEADER") + "\n" + exchange.getIn().getBody())
                .doTry()
                  .unmarshal(csvDataFormat)
                  .process(requestFileService)
                .doCatch(IOException.class)
                  //TODO: custom processing here...
                  .process(exchange -> log.error("caught in dotry: " + exchange.getIn().getBody())).stop()
                .end()//end try/catch
                .choice()
                    .when(simple("${property." + Exchange.SPLIT_COMPLETE + "} == true"))
                    .log(LoggingLevel.INFO, "Finished processing ${file:name}")
                .end();

Solution

  • I had conflicting commons-io version 2.15.0 that was failing the csv unmarshalling silently. after cleaning up the pom with latest commons-io version 2.19.0 - issue was fixed

    I moved the unmarshal(csvDataFormat) out of doTry to see the below exception.

    Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.UnsynchronizedBufferedReader
    
            <dependency>
                <groupId>commons-io</groupId>
                <artifactId>commons-io</artifactId>
                <version>2.19.0</version>
            </dependency>