mapreducehadoop-yarnhadoop2resourcemanager

Resource Manager dies immediately and does not run


I am trying to setup hadoop 2.7.3 single node cluster on my machine. At one time, I successfully ran all the services and ran my custom jar file which worked fine. After stopping and starting the services again later, I found out that Resource Manager is not running. I start the services using:

start-dfs.sh
start-yarn.sh

Doing jps gives:

14755 ResourceManager
13396 DataNode
14884 NodeManager
13606 SecondaryNameNode
15099 Jps
13262 NameNode

which is fine, but after sometime when I run my jar file,it stucks at this:

17/09/08 12:54:59 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
17/09/08 12:55:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/09/08 12:55:02 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

By doing jps again shows Resource manager not running. I am very confused about this, tries lots of solution but not found a single solution.

Here is my /etc/hosts

127.0.0.1       localhost
127.0.1.1       salman-home
# The following lines are desirable for IPv6 capable hosts
#::1     ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters

I have disables the firewall by sudo ufw disable

Here is my yarn-site.xml:

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>127.0.0.1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>127.0.0.1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>127.0.0.1:8031</value>
</property>
</configuration>

mapred-site.xml:

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

core-site.xml:

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost/</value>
    </property>
</configuration>

hdfs-site.xml:

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

Any kind of help will be much appreciated.


Solution

  • It was bit silly of me. That port was be used by InfluxDB. Resource manager tried to acquire it but that tug of war always won by Influx. By stop Influxdb service, I solved this problem.