windowshadoophdfsresourcemanagerdatanode

Datanode not working on Hadoop single node cluster on windows


There are many similar questions on stack overflow but none of them solves my problem.

I'm trying to start my namenode and datanode, of which namenode starts working but datanode fails alongwith resource manager and node manager. Here is the error that shows up:

2021-06-17 15:44:09,513 ERROR datanode.DataNode: Exception in secureMain org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 0, volumes configured: 1, volumes failed: 1, volume failures tolerated: 0 at org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:231) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2799) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2714) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2756) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2900) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2924) 2021-06-17 15:44:09,518 INFO util.ExitUtil: Exiting with status 1: org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 0, volumes configured: 1, volumes failed: 1, volume failures tolerated: 0 2021-06-17 15:44:09,522 INFO datanode.DataNode: SHUTDOWN_MSG:

Here is my hdfs-site.xml:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
     <name>dfs.replication</name>
     <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>C:\Users\username\Documents\hadoop-3.2.1\data\dfs\namenode</value>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>C:\Users\username\Documents\hadoop-3.2.1\data\dfs\datanode</value>
  </property>
  <property>
    <name>dfs.datanode.failed.volumes.tolerated</name>
    <value>0</value>
  </property>
</configuration>

What could be the solution?


Solution

  • The question is answered here:

    https://stackoverflow.com/a/58924939/14194692

    This answer is not accepted on the question but I tried it and it worked. Tada.

    Not deleting my question because none of the question is asked as clearly as this one I believe. I hope it helps other people.

    Cheers.