I am using Spring cloud stream to read a file and split using file splitter and emit each line as a message using DSL style, the file am reading has a header row, just wondering if there is an easy way to skip the header row before/after reading.
Any help is appreciated.
here is how my splitter and integrationFlow looks like:
enter code here
return IntegrationFlows
.from("....")
.split(Files.splitter(true, true)/
.charset(StandardCharsets.UTF_8)
.applySequence(true), //emmit sequenceNumber to header
e -> e.id("fileSplitter")
);
enter code here
IntegrationFlow integrationFlow = integrationFlowBuilder
.<Object, Class<?>>route(Object::getClass, m -> m
.channelMapping(FileSplitter.FileMarker.class, "markers.input")
.channelMapping(String.class, "lines.input"))
.get();
If I read this right you are using one of our OOB apps, the file source: https://github.com/spring-cloud-stream-app-starters/file/blob/master/spring-cloud-starter-stream-source-file/README.adoc and deploying using Spring Cloud Dataflow dsl such as stream create file ----file.consumer.mode=lines --file.directory=/tmp/ | sink
correct?
If so, there's a special header called sequence_number
when you are reading files in the lines mode. You can add a filter in between to drop those messages based on a header expression.