[SOLVED] Regex to extract all the rows from CSV

Regex to extract all the rows from CSV - Apache Nifi

My requirement goes like this.

Need to preserve the incoming flow file (input from CSV file) content in an attribute for further processing as I need to make an HTTP call before making use of the flow file content. So, I'm trying to extract the flow file content (CSV data) into an attribute using "ExtractText" processor with below regex.

([^,]*?),([^,]*),([^,]*)

But this is giving the first line appended with the first column of the second line. But not the whole CSV content from the flow file. How can I get the whole CSV content into an attribute?

The process flow is like below:

GetFile --> ExtractText --> ReplaceText (to frame HTTP body for post method) --> InvokeHTTP -- > AttributeToJSON (convert flowfile from attribute to JSON content) -- > InvokeHTTP (using original CSV data)

Solution

Maybe you are asking how to store the incoming flow content into an Attribute !

For this you would use ExtractText but regex

(?s)(^.*$)

So you will have attributename : (?s)(^.*$).

But - be mindfull of of ExtractText buffersize parameter and group lenght, if you flow content is larger than those values (bufferszie will queue and group lenght will truncate).

Also once into an attribute this would be pushed into your JVM - very slow if they are large.