snowflake-cloud-data-platformazure-data-factory

ADF Script Object using Snowflake Connector doesn't allow script parameters


We use script widgets in ADF to execute Snowflake stored procedures. The script widget connects to our Snowflake database using a Snowflake-linked service. We received a message that we're using legacy drivers and directing us to recreate our linked service. So I faithfully created a new snowflake-linked service, it connects to the Snowflake database, so that part is good.

When I go into a pipeline with a script object using the old driver, it looks something like this: Current Test Pipeline using Old Snowflake Linked Service

{
    "name": "old snowflake connector",
    "properties": {
        "activities": [
            {
                "name": "usp_truncate_table",
                "description": "",
                "type": "Script",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "linkedServiceName": {
                    "referenceName": "snowflake_linked_service",
                    "type": "LinkedServiceReference"
                },
                "typeProperties": {
                    "scripts": [
                        {
                            "parameters": [
                                {
                                    "name": "schema_name",
                                    "type": "String",
                                    "value": "staged",
                                    "direction": "Input"
                                },
                                {
                                    "name": "table_name",
                                    "type": "String",
                                    "value": "brink_order",
                                    "direction": "Input"
                                },
                                {
                                    "name": "jobrunid",
                                    "type": "String",
                                    "value": {
                                        "value": "@pipeline().RunId",
                                        "type": "Expression"
                                    },
                                    "direction": "Input"
                                },
                                {
                                    "name": "pipelinerunid",
                                    "type": "String",
                                    "value": {
                                        "value": "@pipeline().RunId",
                                        "type": "Expression"
                                    },
                                    "direction": "Input"
                                }
                            ],
                            "type": "Query",
                            "text": "call meta.usp_truncate_table(?,?,?,?);"
                        }
                    ],
                    "scriptBlockExecutionTimeout": "02:00:00"
                }
            }
        ],
        "folder": {
            "name": "Source System/misc ad hoc"
        },
        "annotations": []
    }
}

Note that I do have parameters.

When I try to use the new snowflake linked service, I am unable to create any parameters.

The new Script widget looks like this Widget using New Snowflake Linked connector

{
    "name": "new snowflake connector",
    "properties": {
        "activities": [
            {
                "name": "Truncate Staged",
                "type": "Script",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "linkedServiceName": {
                    "referenceName": "snowflake_linked_svc",
                    "type": "LinkedServiceReference"
                },
                "typeProperties": {
                    "scripts": [
                        {
                            "type": "Query",
                            "text": "call meta.usp_truncate_table(?,?,?,?);"
                        }
                    ],
                    "scriptBlockExecutionTimeout": "02:00:00"
                }
            }
        ],
        "folder": {
            "name": "Source System/misc ad hoc"
        },
        "annotations": []
    }
}

This time there isn't an option for Script Parameters.

Has anyone else run into this issue?


Solution

  • I tried to repro the issue and faced similar problem. In New ADF snowflake connector doesn't have the script parameter option is not available. To workaround this you can directly use variables and pipeline parameters directly into the query.

    enter image description here