zipazure-data-factoryaxapta

Azure Data Factory Zip Archive Support

Using Azure Data Factory I have been trying to do the following for to create a D365 Finance and Operations DMF Package API zip file using the native operators in ADF:

This is a relatively trivial task, but it seems the operators within ADF are fairly limited. From what I can see in the documentation:

But it is not available in general for a group of files (for example, using a Binary Sink to make it general purpose).

I know this is achievable this using some code via an Azure Function call for example. However, there is a reluctance to adopt other technologies outside of the core ADF features.

Is there anything I am missing with ADF, or is this just not possible without getting another service to do it?

Solution

  • You can achieve your requirement using Binary datasets in the ADF copy activity.

    After generating the required files in the directory, create Binary datasets for source and target.

    In the source Binary dataset, give your folder location and don't give any file names.

    enter image description here

    These are the starting files that I have in the zipinput folder.

    billing.xml
first.xml
sample1.csv
Second.xml

    In the target dataset, give your folder location and give a name for the zip file <filename>.zip. You can give the Deflate(.zip) compression here.

    enter image description here

    Give these two datasets as the copy activity source and sink and debug the copy activity. Now, it will create required .zip file in the given location.

    enter image description here

    Inside .zip file:

    enter image description here

    You can copy this .zip file to your required location using copy activity with Binary datasets.