hadoopjdbcdriverrdbmshadoop-plugins

How we can use Reducers to insert the resultant data directly into an external RDBMS.what must you be sure to do?? and why

scenario: you are writing a MR job which will use mappers to process data and then use Reducers to insert the resultant data directly into an external RDBMS.what must you be sure to do?? and why

Pre-requsite: 1.Ensure that the database driver is present on the client machine which is submitting the job. 2.Disable speculative execution for the data insert job

Solution

  • 1)If you forgot to disable speculative execution, It is possible that multiple instances of given Reducer could run, which would result extra data than expected into RDBMS. 2)Even we need the database driver for client machine, If you plan to connect to RDBMS from that client , it is not needed.

    So "1" option is correct.

    I got this solution , Can any body Improve this answer or let me correct If any issues. Thank you