I have a file that includes (schemaless) JSON Lines encoded data.
For example:
{"foo" : "abc", "bar" : "def" }
{"foo" : "xyz" }
{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}
I would like to use NIFI to convert this into a JSON array:
[{"foo" : "abc", "bar" : "def" },{"foo" : "xyz" },{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}]
The easiest way to accomplish this in Apache NiFi is to use two ReplaceText processors. In the first, configure as:
\}\s*\{\},\{Regex ReplaceEntire TextThis will remove the line breaks between the tuples and insert commas between them. In the second:
(^.*$)[$1]Regex ReplaceEntire TextThis will add the enclosing brackets around the JSON array. There are other ways to accomplish this with ExecuteScript or JoltTransformJSON processors, but they are more complicated and brittle.