How can I fixed this?
/tmp/hadoop-yarn/staging/ubuntu/.staging/job_1450038005671_0025/job.jar: Under replicated BP-938294433-10.0.1.190-1450037861153:blk_1073744219_3398. Target Replicas is 10 but found 3 replica(s).
I get this when I run hadoop fsck /
in my master node.
I assume I should change a .xml
file in conf or something similar I just don't know which file to change.
Note that dfs.replication
in hdfs-site.xml
is already set to 3. I don't have dfs.replication.max
in my hdfs-site.xml
file.
The replication count for files submitted as part of your job (jars, etc) is controlled by the parameter mapreduce.client.submit.file.replication
(or mapred.submit.replication
in pre 2.4 clusters) in mapred-site.xml. You can adjust this down for clusters that are smaller than 10 nodes, or just ignore the message from fsck.
FWIW, there is a JIRA for this but I doubt it will ever get worked.