I have a file that includes (schemaless) JSON Lines encoded data.
For example:
{"foo" : "abc", "bar" : "def" }
{"foo" : "xyz" }
{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}
I would like to use NIFI to convert this into a JSON array:
[{"foo" : "abc", "bar" : "def" },{"foo" : "xyz" },{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}]
The easiest way to accomplish this in Apache NiFi is to use two ReplaceText
processors. In the first, configure as:
\}\s*\{
\},\{
Regex Replace
Entire Text
This will remove the line breaks between the tuples and insert commas between them. In the second:
(^.*$)
[$1]
Regex Replace
Entire Text
This will add the enclosing brackets around the JSON array. There are other ways to accomplish this with ExecuteScript
or JoltTransformJSON
processors, but they are more complicated and brittle.