nextflow

Azure Batch with Nextflow "Cannot create work-dir"


I am attempting to use Nextflow with Azure batch. I have created the storage account as described in the docs -- a very simple storage account. My profile is:

    az {
        workDir = 'az://bbbbb/scratch'
        process {
            executor = 'azurebatch'
            container = 'aaaaa.azurecr.io/python3:latest'
        }
        azure {
            registry {
                server = 'aaaaa.azurecr.io'
                userName = azure_config["registryAccountName"]
                password = azure_config["registryAccountKey"]
            }
            batch {
                location = 'eastus'
                accountName = azure_config["batchAccountName"]
                accountKey = azure_config["batchAccountKey"]
                autoPoolMode = true
                deletePoolsOnCompletion = true
            }
            storage {
                accountName ="bbbbb"
                accountKey = 'yyyyy'
            }
     }
  }

My nextflow launch is:

nextflow run /home/q/nf-pass/test.nf -profile az

Error:

N E X T F L O W  ~  version 23.10.0
Launching `/home/q/nf-pass/test.nf` [grave_faraday] DSL2 - revision: 63ad770e1c
Cannot create work-dir: /bbbbb/scratch -- Make sure you have write permissions or specify a different directory by using the `-w` command line option

I think my local worktation is attempting to create the blob container az://bbbbb/scratch, but is being denied for some reason. The instructions look pretty simple. Is there a gotcha somewhere ?


Solution

  • The solution was to not refer to the storageaccount name in workdir:

    workDir = 'az://scratch'