pythonmountmlrun

Issue with mount persistent storage in workflow, File not found


When I used fn.apply(mlrun.mount_v3io()) in MLRun function, it worked correctly and I can access to files in mount persistent storage.

import mlrun 
...
fn = mlrun.import_function("test-function.py")

# use mount_v3io() for iguazio volumes
fn.apply(mlrun.mount_v3io())

The issue was, when I used the fn.apply(mlrun.mount_v3io()) in MLRun workflow, I got error message about non exist file (the mount did not work):

Error 1 Metadata file
'rpd.pkl' could not be found v3io://protected/iron/

Did you solve the similar issue in MLRun workflow?


Solution

  • In case of MLRun workflow, you have to use init_functions

    # init functions is used to configure function resources and local settings
    def init_functions(functions: dict, project=None, secrets=None):
        for f in functions.values():
            f.apply(auto_mount())
    
    def kfpipeline():
        taxi_records_csv_path = mlrun.get_sample_path('data/Taxi/yellow_tripdata_2019-01_subset.csv')
        zones_csv_path = mlrun.get_sample_path('data/Taxi/taxi_zones.csv')
        
        # build our ingestion function (container image)
        builder = funcs['taxi'].deploy_step(skip_deployed=True)
        ...
    

    You can see full sample here see title Create the Workflow