I want to copy files from one folder to another folder in data lake using ADF pipelines. Ex : a/b/c/d. TO a/b Here a,b,c,d are folders here I don't want to copy c,d folders .I have to copy the files inside those folders to 'b' folder.
I created a pipeline using Get Metadata , For each and in For Each I used copy activity.But here I am able to copy files with folder itself .I'm failing to remove folders.
I reproduced your scenario follow the below steps:
In my demo container I have nested folders like a/b/c/d under d folder I have 3 files as below.
To copy files from folder to folder I took Get metadata
activity to get list of files from folder.
Dataset for Get Metadata:
Get Metadata settings:
Then I took for-each
activity and passed the output of Get Metadata
activities output to it.
@activity('Get Metadata1').output.childItems
copy activity
inside for each activity and
created source dataset with filename parameter
In file name gave dynamic value as @dataset().filename
In copy activity source gave dynamic value for dataset property filename as @item().name
Now created sink dataset with a/b
directories only
and passed it to sink
Output