azure-data-factoryazure-adf

Check if all files are available in storage - Azure ADF


In Azure data factory, how to check if an array of string(filenames) contains value?

I'm getting file names from get metadata activity and I need to check if all 4 filenames that I have are available in storage account before proceeding.

I'm expecting 4 files into the storage account and I need to check if all the 4 files are available. I need to explicitly check against the file names and not the number of files - this being a requirement

When I try to validate it using the child item from get meta data I am getting error "array elements can only be selected using an integer index." The issue here is that the file could be present at any index in the next load

Is there a better way to validate the filename?

Appreciate your help, thanks in advance


Solution

  • My get meta data output looks like this

     "childItems": [
        {
            "name": "1.py",
            "type": "File"
        },
        {
            "name": "SalesData.numbers",
            "type": "File"
        },
        {
            "name": "file1.txt",
            "type": "File"
        }
    

    ]

    and i used the below expression in set variable activity to check for file names

    @if(
    contains(activity('Get Metadata1').output.childitems,
    json(concat('{"name":"file1.txt"',',','"type":"File"}'))), 
    
    if(
    contains(activity('Get Metadata1').output.childitems,
    json(concat('{"name":"file2.txt"',',','"type":"File"}'))),
    
    if(
    contains(activity('Get Metadata1').output.childitems,
    json(concat('{"name":"2.py"',',','"type":"File"}'))),'yes','no')
    ,'no')
    ,'no')
    

    this checks if my blob has file1.txt, file2.txt and 2.py

    if yes, i am assigning yes to variable else no

    You can use a if condition as well