I'm using Apache NiFi 2.0.0, which unfortunately does not include the PutHDFS processor. My project requires this version of NiFi due to its integration capabilities with Python scripting, so upgrading to a different version with PutHDFS is not an option.
I need to transfer data from NiFi to HDFS, and I'm looking for an alternative way to do this directly within NiFi 2.0.0. Does anyone have suggestions for accomplishing this? Here are some details of what I’m considering:
Python Scripted Processors: Given that NiFi 2.0.0 has Python scripting support, is there a way to leverage this to create a custom solution that writes to HDFS?
Other Processors or Workarounds: If there are alternative processors or methods to get data into HDFS (e.g., using ExecuteStreamCommand to run HDFS CLI commands or any HTTP-based processors), I’d be interested in those as well.
Any guidance on how to handle this or examples would be greatly appreciated!
You can find more details on what components have been removed in NiFi 2 here: https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features
Also for reference: https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
For HDFS components specifically, there are not packaged in the convenience binary file (ie. nifi-2.0.0-bin.zip) but the components are still built and made available through Maven repositories. You can download the appropriate NARs as needed: