When I run a snakemake workflow locally with snakemake --cores 1
, everything works fine. However I need to use SLURM. I tried the--cluster
command that I've seen in the documentation and in nearly every tutorial:
snakemake: error: unrecognized arguments: --cluster
I also tried --slurm
, not recognized either. --cluster-config
also doesn't appear to be understood by Snakemake.
Looking for some explanation I found that apparently Snakemake 8 made breaking changes. It looks like now things are like this:
New designed: Now if you want to use --cluster CMD, please use --executor cluster-generic --cluster-generic-submit-cmd CMD instead.
Note you should install cluster-generic using command pip install snakemake-executor-cluster-generic
Neither pip nor conda knows a snakemake-executor-cluster-generic
package. How do I work with slurm, then?
I ran into a very similar issue. I used to do something like this for a SGE/UGE cluster:
snakemake \
--cluster-config cluster.json \
-j 1 \
--cluster 'qsub -q all.q -l h_vmem={cluster.mem} -pe smp {cluster.num_cores} -V -cwd'
Where the contents of cluster.json
were:
{
"__default__": {
"mem": "8G",
"num_cores": "{threads}",
"jobname": "{rule}"
}
}
In Snakemake 8, it appears you first need to install a new plugin by going:
pip install snakemake-executor-plugin-cluster-generic
The --cluster-config
argument is no longer available. So the first command becomes:
snakemake \
--executor cluster-generic \
--cluster-generic-submit-cmd 'qsub -N {rule} -q all.q -l h_vmem=8G -pe smp {threads} -V -cwd'
This logic should work for other clusters (e.g. SLURM).