sql-serverhadoophdfssqoop

Sqoop export to SQL Server: schemas?


I would like to export data in HDFS to a SQL Server table in the schema my_schema.

I tried --schema like the import command:

sqoop export \
--libjars /opt/mapr/sqoop/sqoop-1.4.6/lib/sqljdbc4.jar \
--connect "jdbc:sqlserver://MY-SERVER-DNS;database=my_db;" \
--schema "myschema" \
--table "my_table" \
--export-dir /path/to/my/hdfs/dir

ERROR tool.BaseSqoopTool: Unrecognized argument: --schema

And --table "schema.table"

sqoop export \
--libjars /opt/mapr/sqoop/sqoop-1.4.6/lib/sqljdbc4.jar \
--connect "jdbc:sqlserver://MY-SERVER-DNS;database=my_db;" \
--table "my_schema.my_table" \
--export-dir /path/to/my/hdfs/dir

INFO manager.SqlManager: 
Executing SQL statement: SELECT t.* FROM [my_schema.my_table] AS t WHERE 1=0 

ERROR manager.SqlManager: Error executing statement: 
com.microsoft.sqlserver.jdbc.SQLServerException:
Invalid object name 'my_schema.my_table'.

Is there any way to do this with sqoop? Or another technology?

EDIT:

sqoop export \
--libjars /opt/mapr/sqoop/sqoop-1.4.6/lib/sqljdbc4.jar \
--connect "jdbc:sqlserver://MY-SERVER-DNS;database=my_db;schema=my_schema;" \
--table "my_table" \
--export-dir /path/to/my/hdfs/dir

INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [my_table] AS t WHERE 1=0
16/07/25 10:46:21 ERROR manager.SqlManager: 
Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException:
Invalid object name 'my_table'.

This is not recognizing the schema.


Solution

  • provide schema in connection string:

    jdbc:sqlserver://MY-SERVER-DNS;databaseName=my_db;schema=my_schema;

    so your export sqoop command should be:

    sqoop export \
    --libjars /opt/mapr/sqoop/sqoop-1.4.6/lib/sqljdbc4.jar \
    --connect "jdbc:sqlserver://MY-SERVER-DNS;databaseName=my_db;schema=my_schema;" \
    --table "my_table" \
    --export-dir /path/to/my/hdfs/dir
    

    ALSO:

    Custom schemas are supported for both import and export job - from syntax in sqoop guide, --schema should be passed as -- --schema:

    so your export sqoop command should be:

    sqoop export \
    --libjars /opt/mapr/sqoop/sqoop-1.4.6/lib/sqljdbc4.jar \
    --connect "jdbc:sqlserver://MY-SERVER-DNS;databaseName=my_db;" \
    --export-dir /path/to/my/hdfs/dir
    --table "my_table" \
    -- --schema my_schema \