I am attempting to run Python from inside a Nextflow pipeline launched via Nextflow-Tower. I forked the nextflow-io/hello repo and made sure I could launch it successfully from my own repo. What I am doing wrong? If there is a simple python pipeline repo that can be launched(foolproof) using Nextflow-Tower that would be helpful.
My dir structure is:
root
--main.nf
--nextflow.config
--bin
----test.py
nextflow.config
process.container = "us-east1-docker.pkg.dev/xxxx-yyyy/pass/python3:latest"
docker.enabled = true
main.nf
#!/usr/bin/env nextflow
nextflow.enable.dsl=2
process sayHello {
input:
val x
output:
stdout
script:
"""
echo '$x world!'
"""
}
process processPython {
input:
val y
output:
stdout
script:
"""
#!/usr/bin/env python3
test.py
"""
}
workflow {
Channel.of('Bonjour', 'Ciao', 'Hello', 'Hola') | sayHello | view
Channel.of('foo', 'bar', 'cluster', 'frack') | processPython | view
}
test.py
import sys
print(f"Python Version: {sys.version}")
Error report
Error executing process > 'processPython (1)'
Caused by:
Process `processPython (1)` terminated with an error exit status (127)
Command executed:
#!/usr/bin/env python3
test.py
Command exit status:
127
Command output:
(empty)
Command error:
/usr/bin/env: 'python3': No such file or directory
Work dir:
gs://nf-tower-6897ad32-d3d4-4bda-ade1-97d25c0e680c/scratch/4wCvYTX4GV6S3k/98/707cf12d829d888f4f5179ee1ae55e
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
The solution was to include shebang IN test.py:
#!/usr/bin/env python3
import sys
print(f"Python Version: {sys.version}")
process:
process processPython {
container = 'us-east1-docker.pkg.dev/xxxx-yyyy/pass/python3:latest'
input:
val y
output:
stdout
script:
"""
python3 --version
test.py
"""
}