I am deploying a Storm topology on my machine configured as a single local cluster. I have configured the conf/storm.yaml to use the storm.scheduler: "org.apache.storm.scheduler.resource.ResourceAwareScheduler". The topology is deployed successfully. However I am receiving an error from the Sigar library which says that cannot get the Process ID to use the CPUMetric on my topology. Here is the configuration of my topology to get metrics:
config.registerMetricsConsumer(LoggingMetricsConsumer.class);
Map<String, String> workerMetrics = new HashMap<String, String>();
workerMetrics.put("CPU", "org.apache.storm.metrics.sigar.CPUMetric");
config.put(Config.TOPOLOGY_WORKER_METRICS, workerMetrics);
I already copy the sigar-1.6.4.jar and storm-metrics-1.2.2.jar libraries to the apache-storm/lib folder. Here is the error:
2019-03-13 16:24:45.920 o.a.s.util Thread-10-__system-executor[-1 -1] [ERROR] Async loop died!
java.lang.UnsatisfiedLinkError: org.hyperic.sigar.Sigar.getPid()J
at org.hyperic.sigar.Sigar.getPid(Native Method) ~[sigar-1.6.4.jar:?]
at org.apache.storm.metrics.sigar.CPUMetric.<init>(CPUMetric.java:38) ~[storm-metrics-1.2.2.jar:1.2.2]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_191]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_191]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_191]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_191]
at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_191]
at org.apache.storm.utils.Utils.newInstanceImpl(Utils.java:198) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.utils.Utils.newInstance(Utils.java:192) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.utils.Utils.newInstance(Utils.java:185) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.metric.SystemBolt.registerMetrics(SystemBolt.java:150) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.metric.SystemBolt.prepare(SystemBolt.java:143) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.daemon.executor$fn__10795$fn__10808.invoke(executor.clj:803) ~[storm-core-1.2.2.jar:1.2.2]
at org.apache.storm.util$async_loop$fn__553.invoke(util.clj:482) [storm-core-1.2.2.jar:1.2.2]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
The native part of Sigar is not on the classpath of the worker for some reason. The native files are found inside storm-metrics.jar in the resources directory.
Why are you manually copying storm-metrics to storm/lib? The easiest way to ensure your topology dependencies are present would be making a fat jar using the maven-shade-plugin. Take a look at how storm-starter does it https://github.com/apache/storm/blob/master/examples/storm-starter/pom.xml#L153.
I would check the log of the worker to verify that the storm-metrics.jar file is on the classpath of the worker process. The worker prints its classpath very early during boot, as does the supervisor.
You mention that you're running on a single local cluster. I'm not sure if you mean a single-node install of Storm, or if you're using LocalCluster (or equivalently the storm local command). If you mean you're using LocalCluster, you will need to add storm-metrics as a dependency of your topology project.