javaspring-integration-dslspring-dsl

Read file line by line using spring integration dsl


I'm trying to read file, but I need to split it line by line using Spring Integration DSL. What I need to add into my integration flows to make it works

Message source

 @Bean
    public MessageSource<File> sourceDirectory() {
        FileReadingMessageSource messageSource = new FileReadingMessageSource();
        messageSource.setDirectory(new File(fileSource));
        return messageSource;
    }

Filter

@Bean
    public GenericSelector<File> onlyCSVs() {
        return (file) -> file.getName().endsWith(".csv");
    }

File transformer

 @Bean
    public FileToStringTransformer fileToStringTransformer() {
        return new FileToStringTransformer();
    }

Integration flow

@Bean
    public StandardIntegrationFlow integrationFlow() {
        return IntegrationFlows
                .from(sourceDirectory(), configurer -> configurer.poller(Pollers.fixedDelay(10000)))
                .channel("fileInputChannel")

                .filter(onlyCSVs())
                .transform(fileToStringTransformer())
                .handle(m -> System.out.println((String.valueOf(Math.random()) + m.getPayload())))
                .get();
    }

Solution

  • after your filetoString transfomer, I would add one another custom transformer which takes the string and makes an array like this.

    String.split("[\\r\\n]+")
    

    it already removes empty lines, then after that, I would add an .split() to the flow so it creates a message for every line, .split() already works with iterators, in case just transform the array to a list and you're done.

    it's something like:

    @Bean
    public StandardIntegrationFlow integrationFlow() {
        return IntegrationFlows
                .from(sourceDirectory(), configurer -> configurer.poller(Pollers.fixedDelay(10000)))
                .channel("fileInputChannel")
    
                .filter(onlyCSVs())
                .transform(fileToStringTransformer())
                .transform(GenericMessage.class, message -> message.getPayload().split("[\\r\\n]+"))
                .split()
                .handle(m -> System.out.println((String.valueOf(Math.random()) + m.getPayload())))
                .get();
    }