linuxhadoopbigdatahbaseapache-zookeeper

HBase Shell - org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet


I am trying to set up distributed HBase on 3 nodes. I have already set up hadoop, YARN ZooKeeper and now HBase but when I launch hbase shell and run the simplest command for example status or list I get the exception:

HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.4.8, rf844d09157d9dce6c54fcd53975b7a45865ee9ac, Wed Oct 27 08:48:57 PDT 2021
Took 0.0014 seconds
hbase:001:0> status

ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
        at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2817)
        at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1205)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:354)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:334)

For usage try 'help "status"'

Took 9.0800 seconds

My Hadoop Master JPS:

3443 NameNode
13987 Jps
2503 QuorumPeerMain
3754 SecondaryNameNode
4010 ResourceManager
4749 HMaster

and both Nodes JPS:

2135 QuorumPeerMain
6327 HRegionServer
3115 NodeManager
2894 DataNode
6670 Jps

I launch my own ZooKeeper with following zoo.cfg:

tickTime=2000
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/data
clientPort=2181
initLimit=10
syncLimit=10
server.1=0.0.0.0:2888:3888
server.2=OpenCitation2:2888:3888
server.3=OpenCitation3:2888:3888
4lw.commands.whitelist=stat, ruok, conf, isro



hadoopuser@OpenCitation1:~$ echo ruok | nc OpenCitation1 2181
imok
hadoopuser@OpenCitation1:~$ echo ruok | nc OpenCitation2 2181
imok
hadoopuser@OpenCitation1:~$ echo ruok | nc OpenCitation3 2181
imok

Hbase-site.xml:

      <property>
        <name>hbase.rootdir</name>
        <value>hdfs://OpenCitation1:9000/hbase</value>
      </property>
      <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
      </property>
      <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/usr/local/zookeeper/data</value>
      </property>
      <property>
        <name>hbase.unsafe.stream.capability.enforce</name>
        <value>false</value>
      </property>
      <property>
        <name>hbase.tmp.dir</name>
        <value>/usr/local/HBase/var</value>
      </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
    </property>

Hbase-env.sh:

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
export HBASE_MANAGES_ZK=false

Hadoop version:

hadoopuser@OpenCitation1:~$ hadoop version
Hadoop 3.3.1
Source code repository https://github.com/apache/hadoop.git -r a3b9c37a397ad4188041dd80621bdeefc46885f2
Compiled by ubuntu on 2021-06-15T05:13Z
Compiled with protoc 3.7.1
From source with checksum 88a4ddb2299aca054416d6b7f81ca55
This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-3.3.1.jar

Hbase version:

hbase:001:0> version
2.4.8, rf844d09157d9dce6c54fcd53975b7a45865ee9ac, Wed Oct 27 08:48:57 PDT 2021
Took 0.0005 seconds

Also I tried turning safemode:

hadoopuser@OpenCitation1:~$ hdfs dfsadmin -safemode get
Safe mode is OFF

but it didn't work either.

I need help because I am really running out od ideas...

Thanks in advance


Solution

  • UPDATE:

    I have solved the issue by adding the following property to the hbase-site.xml:

    <property>
      <name>hbase.wal.provider</name>
      <value>filesystem</value>
    </property>
    

    Now it runs:

    hadoopuser@OpenCitation1:~$ hbase shell
    HBase Shell
    Use "help" to get list of supported commands.
    Use "exit" to quit this interactive shell.
    For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
    Version 2.4.8, rf844d09157d9dce6c54fcd53975b7a45865ee9ac, Wed Oct 27 08:48:57 PDT 2021
    Took 0.0014 seconds
    hbase:001:0> status
    1 active master, 0 backup masters, 2 servers, 0 dead, 1.0000 average load
    Took 0.4730 seconds
    hbase:002:0>
    

    Maybe it will help someone