javahadoophbasemetricsganglia

How to enable new metrics on Ganglia from Hbase?


How can I enable new metrics to Ganglia from Hbase 0.98?

for example: I have enabled the default metrics to monitoring from ganglia, but I would like to enable a specific metric in my cluster like this "blockCacheExpressCachingRatio" or others more.

Thanks


Solution

  • By "Default metric", I assume you are indicating OS specific metrics like cpu steal time, memory free, memory used, etc. These metrics are enabled by default in ganglia while you are done with ganglia installation.

    Now, from my level of experience, I have learned that HBASE emits all related metrics directly to the server on which gmetad is installed. Yes, it is true that these hbase related metrics are not first delivered to the gmond installed on the same machine. This is true for hadoop based metrics also.

    Note: ganglia-server is the FQDN of the machine on which gmetad is installed.

    To be more precise, gmond installed on the ganglia-server never asks or digs any specific metric from Hbase. gmond of ganglia-server just sits and keep listening to the default port 8649 and collect whatever metrics arrive. It could be sent by hbase, hadoop or gmond installed on any remote server.

    So, from the ganglia side, one is helpless to be selective on such metrics. Graphs are built by rrdtool for whatever metrics are collected by gmond on ganglia-server.

    So, one may assume that metrics could be chosen by configuring from the source where it is emitted. HBase is the source here. But, unfortunately, as far as I have known, this could not be changed. You can simply enable hbase related metrics by inserting following lines in the file "hadoop-metrics2-hbase.properties", which is found in "conf" directory under the HBASE_HOME directory:

    *.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
    *.sink.ganglia.period=10
    hbase.sink.ganglia.period=10
    hbase.sink.ganglia.servers=ganglia-server:8649
    

    Note: Restart hbase after changing hadoop-metrics2-hbase.properties.

    Please note that this has been tested with hbase version 0.98.7. I Hope it helps you.