I'm trying to get maximum network performance with CDH5 Cloudera with 10 nodes with 1 MapReduce TaskTracker but can't seem to get the traffic to produce the traffic I need. I have 40G nic cards and currently only getting 10G at the most with is really low.
When using 1 MapReduce TaskTracker how can I produce better performance and network throughput?
Running -- 10 nodes RedHat 6.6 Current Nic driver CDH5 Kernel: 3.18.1
I've decommissioned 9 MapReduce TaskTracker nodes but still not getting good results.
I finally figured out that you really don't need a task tracker on data nodes for network dfsio test so I decommissioned all datanode "task trackers" and put the task tracker on the name node and now I have both write/tx and read/rx traffic now which is what I wanted for network throughput data.