hadoopmapreducesqoophadoop-yarnsqoop2

Sqoop - Binding to YARN queues

So with mapreduce v2 you can use binding to certain YARN queues to manage resources and prioritization. Basically by using

"hadoop jar /xyz.jar -D mapreduce.job.queuename=QUEUE1 /input /output" which works perfectly.

How can integrate Yarn queue binding with Sqoop when you run a sqoop query?

ie. sqoop import \ --connect 'jdbc://server' \ --target-dir \ and what ?

Solution

  • Use the same method for Sqoop as well i.e 

    sqoop import -Dmapreduce.job.queuename=NameOfTheQueue\ --connect 'jdbc://server' \ --target-dir \

    Note that, the mapreduce.job.queuename parameter should be used after the import but before the connect or other options.