centosapache-zookeeperapache-stormnimbus

Error while Running Multiple Supervisors on Apache Storm UI


We have zookeeper running on one machine , Nimbus on second and then two supervisors(workers) running on different machines.

Zookeper is running on windows 7 and all others are running over Cent OS.

Now Problem is that when we run storm UI on machine running nimbus, it displays only single superviser (which randomly changes between the two supervisors on refreshing the page).

How to display both of them on UI simultaneously?

#Zookeeper ip = 10.135.155.133
#Nimbus ip = 10.135.158.22
#Supervisor 1 ip = 10.135.156.63
#supervisor 2 ip = 10.135.156.162

Below is zoo.cfg file of zookeper (on first machine)

tickTime=2000

initLimit=10

syncLimit=5

dataDir=D:\\tmp\\zookeeper

clientPort=2181

Below is Storm.yaml file coniguration running nimbus (on second machine)

storm.zookeeper.servers:
    - "10.135.155.133"

storm.local.dir: "/storm/apache-storm-1.1.0/lib/" 

nimbus.host: "10.135.158.22"

Below is Storm.yaml file configuration running supervisor 1 (on third machine)

storm.zookeeper.servers:
    - "10.135.155.133"



supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

 storm.local.dir: "/storm/apache-storm-1.1.0/new" 

 nimbus.host: "10.135.158.22"

Below is Storm.yaml file configuration running supervisor 2 (on fouth machine).

storm.zookeeper.servers:
    - "10.135.155.133"



supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

storm.local.dir:"/storm/apache-storm-1.1.0/new 2" 

nimbus.host: "10.135.158.22"

Solution

  • Symptoms:

    Some supervisor processes are missing from the Storm UI.

    List of supervisors in Storm UI changes on refreshes

    Solutions:

    Make sure the supervisor local dirs are independent (e.g., not sharing a local dir over NFS) Try deleting the local dirs for the supervisors and restarting the daemons.

    Supervisors create a unique id for themselves and store it locally. When that id is copied to other nodes, Storm gets confused.