hadoopmapreducehadoop-yarn

Spark Driver vs MapReduce Driver on YARN


I know in spark you can run the driver program on the client machine if you specify `yarn-client` deployment mode. Or you can run it on a random machine in the cluster if you specify `yarn-cluster` mode.

Now my question is: In MapReduce applications where does the driver program runs? client or cluster?

Thanks,

Solution

  • When you use hadoop jar in the terminal, that terminal session is the driver client

    If you use Oozie, instead, for example, the driver will be deployed in the cluster