azureazure-data-factorydynamic-columns

Azure Data Factory Get Metadata activity returning "(404) not found" error when getting column count


I am trying to implement a Get Metadata activity to return the column count of files I have in a single blob storage container.

Get Metadata activity is returning this error: Error

I'm fairly new to Azure Data Factory and cannot solve this. Here's what I have:

Dataset:Source dataset

Name- ten_eighty_split_CSV

Connection- Blob storage

Schema- imported from blob storage file

Parameters- "FileName"; string; "@pipeline().parameters.SourceFile"

Pipeline:

Name: ten eighty split

Parameters: "SourceFile"; string; "@pipeline().parameters.SourceFile"

Settings: Concurrency: 1

Get Metadata activity: Get Metadata

Only argument is "Column count"

Throws the error upon debugging. I am not sure what to do, (404) not found is so broad I could not ascertain a specific solution. Thanks!


Solution

  • enter image description here

    enter image description here

    enter image description here

    The following is the trigger JSON that I created for my container:

    {
        "name": "trigger1",
        "properties": {
            "annotations": [],
            "runtimeState": "Started",
            "pipelines": [
                {
                    "pipelineReference": {
                        "referenceName": "pipeline1",
                        "type": "PipelineReference"
                    },
                    "parameters": {
                        "sourceFile": "@triggerBody().fileName"
                    }
                }
            ],
            "type": "BlobEventsTrigger",
            "typeProperties": {
                "blobPathBeginsWith": "/data/blobs/",
                "blobPathEndsWith": ".csv",
                "ignoreEmptyBlobs": true,
                "scope": "/subscriptions/b83c1ed3-c5b6-44fb-b5ba-2b83a074c23f/resourceGroups/<user>/providers/Microsoft.Storage/storageAccounts/blb1402",
                "events": [
                    "Microsoft.Storage.BlobCreated"
                ]
            }
        }
    }