Trying to setup Flume on edge node, I was checking through many blogs but haven't got much idea as most of them are referring a single node cluster, can someone suggest it is good idea to setup on edge node or this will be on server where HDFS or any worker node setup (Data-node), if yes then what will be configuration to setup this on Edge node.
If this is a production environment, its a good idea to avoid NameNode server(s), Resource Manager server(s), journal nodes and DataNodes. That leaves you with edge node.
The process would be to:
1) Install Hadoop client.
2) Install Flume
3) Configure the flume in flume.conf file (or whatever name you want to give). You can find many sample configurations online.
4) Make monitoring type = http for quick check of performance data.
5) Open the ports for Sources and Sinks.
5) Start the agent.
6) Check the agent log to see all components started.
7) Try sending some sample data and check if it reaches destination.
8) Debug any failures.
Let me know if you need more information.