arraysjsonapacheapache-nifischemaless

Convert JSON Lines to JSON array using Apache Nifi


I have a file that includes (schemaless) JSON Lines encoded data.

For example:

{"foo" : "abc", "bar" : "def" }
{"foo" : "xyz" }
{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}

I would like to use NIFI to convert this into a JSON array:

[{"foo" : "abc", "bar" : "def" },{"foo" : "xyz" },{"foo" : "ghi", "bar" : "jkl", "name" : "The Dude"}]

Solution

  • The easiest way to accomplish this in Apache NiFi is to use two ReplaceText processors. In the first, configure as:

    This will remove the line breaks between the tuples and insert commas between them. In the second:

    This will add the enclosing brackets around the JSON array. There are other ways to accomplish this with ExecuteScript or JoltTransformJSON processors, but they are more complicated and brittle.