mulemule-studioanypoint-studiofilesplittingndjson

ND-JSON Split in SFTP

I have a large ND-JSON file in SFTP (~20K lines). Is there a way to generate sub files out of this (~500 lines each) and place in another folder in SFTP?

Does Mule 4 has the capability to split a large file and write in SFTP? Or is there a need for a Java component?

Please advise.

Solution

  • If the input file is parsed as NDJSON, you can use the DataWeave function divideBy() to separate the array read from the file into subarrays of n elements.

    Example:

    %dw 2.0
output application/java
import * from dw::core::Arrays
---
payload divideBy 500

    Then you should be able to use a to process each segment and output an NDJSON file inside.