azure-data-factoryazure-powershell

Create Pipeline via Azure Powershell and pipelineReturnValue activity is not properly created


I try to run this command in Azure Powershell

Set-AzDataFactoryV2Pipeline -Force -ResourceGroupName MY_DEV_RESOURCE_GRP -DataFactoryName MYADF001 -Name PL_PROCESS_API -DefinitionFile ./PL_PROCESS_API.json

Inside a PL_PROCESS_API.json file is containing this content

...{
            "name": "Activity_Return",
            "type": "SetVariable",
            "dependsOn": [
                {
                    "activity": "Process API data",
                    "dependencyConditions": [
                        "Succeeded"
                    ]
                }
            ],
            "policy": {
                "secureOutput": false,
                "secureInput": false
            },
            "userProperties": [],
            "typeProperties": {
                "variableName": "pipelineReturnValue",
                "value": [
                    {
                        "key": "strCommaSeparatedString",
                        "value": {
                            "type": "Expression",
                            "content": "@activity('Process API data').output.Holiday"
                        }
                    }
                ],
                "setSystemVariable": true
            }
        }...

When I run this command the sections of pipelineReturnValue in type properties in ADF when the command run successfully will be like this

"typeProperties": {
                "variableName": "pipelineReturnValue",
                "value": "[{\"key\":\"strCommaSeparatedString\",\"value\":{\"type\":\"Expression\",\"content\":\"@length(activity('Process API data').output.Holiday)\"}}]"
            }

I would like to know how to fix this?


Solution

  • I have tried the same PowerShell script with below JSON which is similar as your pipeline from my end.

    {
        "name": "test2",
        "properties": {
            "activities": [
                {
                    "name": "Activity_Return",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "Process API data",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "policy": {
                        "secureOutput": false,
                        "secureInput": false
                    },
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "pipelineReturnValue",
                        "value": [
                            {
                                "key": "strCommaSeparatedString",
                                "value": {
                                    "type": "Expression",
                                    "content": "@activity('Process API data').output.data"
                                }
                            }
                        ],
                        "setSystemVariable": true
                    }
                },
                {
                    "name": "Process API data",
                    "type": "WebActivity",
                    "dependsOn": [],
                    "policy": {
                        "timeout": "0.12:00:00",
                        "retry": 0,
                        "retryIntervalInSeconds": 30,
                        "secureOutput": false,
                        "secureInput": false
                    },
                    "userProperties": [],
                    "typeProperties": {
                        "method": "GET",
                        "url": "https://reqres.in/api/users?page=2"
                    }
                }
            ],
            "variables": {
                "var1": {
                    "type": "String"
                }
            },
            "annotations": []
        },
        "type": "Microsoft.DataFactory/factories/pipelines"
    }
    

    The pipeline got created with correct expression as shown below.

    enter image description here

    I have tried by changing some code in the pipeline JSON and the only time I got the similar JSON string for the return variable is when I had set the "setSystemVariable" property to false in the typeProperties of the Activity_Return activity.

    This is the activity JSON for your reference:

    {
                    "name": "Activity_Return",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "Process API data",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "policy": {
                        "secureOutput": false,
                        "secureInput": false
                    },
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "pipelineReturnValue",
                        "value": [
                            {
                                "key": "strCommaSeparatedString",
                                "value": {
                                    "type": "Expression",
                                    "content": "@activity('Process API data').output.data"
                                }
                            }
                        ],
                        "setSystemVariable": false 
                    }
                }
    

    Similar Result:

    enter image description here

    When the setSystemVariable property is false, the pipeline variable will not be considered as a pipeline return variable and the expression for that will be considered as a string. So, make sure the setSystemVariable property is set to true in the pipeline JSON.