hadoophdfsdistcp

How to change hadoop distcp staging directory

when I ran command 

hadoop distcp -update hdfs://path/to/a/file.txt hdfs://path/to/b/

I got an Java IOException:

java.io.IOException: Mkdirs failed to create /some/.staging/directory

However, I don't want to use "/some/file/path" as a temporary staging directory, how can I change it to another directory I want?

Solution

  • It's controlled via yarn.app.mapreduce.am.staging-dir (defaulting to /user in my install).