azure-data-factory

ADF how to set timeout dynamically in stored procedure


I am trying to run an ADF pipeline which just runs a stored procedure on the database, is there a way to set the Timeout value dynamically under the General tab of stored procedure? I have no way to manually change the move procedure as it's written by some other team and they are not giving any access to it. I just want it to be dynamically set in case some stored procedure takes more time than required gets killed. Stored Procedure General Tab


Solution

  • Currently, it is an out of box feature in ADF. I have tried to set the timeout in connection string in the linked service using linked service parameters in its JSON like below.

    enter image description here

    but it's not working as expected. I suggest you once try and check whether it's working or not.

    As a workaround for this problem, you can use another pipeline in ADF.

    Follow the below pipeline design for your pipeline. For this, use Pipeline Runs REST APIs.

    - Web activity - POST with Create Run REST API and pass current pipeline run id as parameters in the web activity body. This will trigger pipeline2.
    - Your stored procedure activity.
    

    Pipeline2 design:

    Create a string parameter in the pipeline. The name should be same as the parameter passed from web activity from above pipeline. This will capture above pipeline run id during the pipeline run.
    - wait activity - set your timeout here.
    - web activity - Get the status of the above pipeline using Get REST API. Use the parameter for passing the above pipeline run id into the REST API URL.
    - if activity - Check the status of the above pipeline from the body of the web activity is InProgress or not.
        - Web activity inside True activities of if - If it is still InProgress, cancel the pipeline run using  Cancel REST API POST. Use parameter for the pipeline run id.
    

    This will cancel the pipeline run when the above pipeline is still in InProgress.

    NOTE: This approach will only work when your pipeline contains only stored procedure activity.