I am on CDH 5.1.2, I am seeing this error with one of the datanode pausing often. i see this from logs.
WARN org.apache.hadoop.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 12428ms
GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=12707ms
Any Idea why i am seeing this? once a while hdfs capacity is dropping by one node.
GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=12707ms
You're experiencing a long GC pause with the CMS collector.
To investigate further you should turn on GC logging via -Xloggc:<path to gc log file> -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCDetails and in case you're on java 7 also add -XX:+PrintGCCause.
GCViewer can help visualizing the logs.
Once you've found the cause you can try adjust CMS to avoid those pauses. For starters, there is the official CMS tuning guide.