azure-data-factorycustom-activity

Executing Batch service in Azure Data factory using python script


Hi i've been trying to execute a custom activity in ADF which receives csv file from the container (A) after further transformation on the data set, transformed DF stored into another csv file in a same container (A). I've written the transformation logic in python and have it stored in the same container (A). Error raises here, when i execute the pipeline it returns an error *can't find the specified file * Nothing wrong in the connections, Is anything wrong in batch Account or pools!! can anyone tell me where to place the python script..!!!


Solution

  • Install azure batch explorer and make sure to choose proper configuration for virtual machine (dsvm-windows) which will ensure python is already in place in the virtual machine where your code is being run.

    This video explains the steps

    https://youtu.be/_3_eiHX3RKE