azureazure-sql-databaseazure-data-factory

I want to fetch Azure SQL table data from ADF to use as input in copy activity

I have 4 SQL tables.

  1. Employee(which contains employee information)
  2. Engineers( same structure as EMployee but no input)

3)Student( same structure as EMployee but no input)

4)Metadata ( contains the source and destination information such as source: Employee destination: Student)

My goal is to create an ADF pipeline that will fetch metadata table information (source and destination) and will perform copy activity from the source to destination as per data in the Metadata table.

Employee Table

enter image description here

Metadata Table

Flag Source destination 2 dbo.Employee dbo.Engineer

Engineer Table

ID First_name Last_name

I want my ADF pipeline to fetch the source and destination data from Metadata table and perform copy activity from source to destination as per mentioned in Metadata table.

I have tried the below approaches but not succeeded yet.

getMetadata---> copyActivity [ not able to fetch the metadata table data]

getmetadata--->foreach loop----> copy activity [ again for able to fetch the data]

Solution

  • You should use Lookup active not Get Metadata active.

    According you problem,, please follow this workaround:

    1. Using Lookup active to get the content of the table Metadata. enter image description here
    2. For each active to loop the each row of Metadata: enter image description here
    3. Foreach inner actives, create the copy active which source and sink all set with foreach item. For example: enter image description here enter image description here

    HTH.