Can I use custom ordering on GetMetadata activity in Azure Data Factory?

I am trying to set up metadata driven ETL to load incoming text files to my warehouse. The issue that I am running into is that the files represent fact and dimension tables that have foreign key constraints on them. Because of this I need to load all of the dim tables first before loading the fact tables. I am using the GetMetadata activity in ADF to get a list of child items (file names) from the staging folder. I am then using a ForEach activity to load each file to the corresponding table. However, I have not been able to find a way to order the list returned by the GetMetadata activity to ensure that the dim tables are loaded before the fact tables. Is there a way to enforce an order in which the files are loaded? I am sure there are plenty of work arounds, like separating the dim and fact files into separate folders and then loading them separately. I am just looking for if there is a way to enforce order after the GetMetadata activity or inside of the ForEach activity.

Yes, you are right we can not order or sort the child items from GetMetadata activity, so below are the available work around.

1. If you can hard code the file names then create an array with filenames starting from dim table names followed by fact table names.
2. If there are too many files that you cannot hard code , then you can create 2 array variables where you append values to it according to the file names using ForEach and If-Else activity.

This applies only if you have file names like below pattern Dim_cust.txt,Dim_sale.txt,Fact_table1.txtetc.

Below is the flow.

1. Create 2 array variables.

2. GetMetadata Activity.

3. ForEach activity

3. Inside for each add condition and Append variable activity.

Expression: @contains(item().name,'dim') You alter above accordingly.

Next, Append Variable - True

Append Variable - False

Next, again use ForEach activity for Dim tables followed by Fact tables like below.

Here, i used set variable to show the output but in your case, you load the table.

Output:

First, it took Dim file names.

and next 1 fact file name.