I am trying to build a workflow in snakemake, but as soon as I use conda, snakemake immediately reports an error. I wrote a simple example to illustrate this error.
Here is a snakefile file:
rule a_conda_rule:
shell:
"which htop"
Everything seems fine after execution:
$ snakemake -j1
Assuming unrestricted shared filesystem usage.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count
------------ -------
a_conda_rule 1
total 1
Select jobs to execute...
Execute 1 jobs...
[Fri Apr 12 23:59:21 2024]
localrule a_conda_rule:
jobid: 0
reason: Rules with neither input nor output files are always executed.
resources: tmpdir=/tmp
$HOME/local/htop/bin/htop
[Fri Apr 12 23:59:21 2024]
Finished job 0.
1 of 1 steps (100%) done
Complete log: .snakemake/log/2024-04-12T235921.291193.snakemake.log
(snakemake)
But once you add a line of conda:
rule a_conda_rule:
conda: "environment.yml"
shell:
"which htop"
It immediately becomes like this:
$ snakemake -j1 --use-conda
Assuming unrestricted shared filesystem usage.
Building DAG of jobs...
Traceback (most recent call last):
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/cli.py", line 2068, in args_to_api
dag_api.execute_workflow(
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/api.py", line 589, in execute_workflow
workflow.execute(
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1122, in execute
self.dag.create_conda_envs()
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/dag.py", line 420, in create_conda_envs
env.create(self.workflow.dryrun)
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 393, in create
pin_file = self.pin_file
^^^^^^^^^^^^^
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake_interface_common/utils.py", line 33, in __get__
value = self.method(instance)
^^^^^^^^^^^^^^^^^^^^^
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 103, in pin_file
f".{self.conda.platform}.pin.txt"
^^^^^^^^^^
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake_interface_common/utils.py", line 33, in __get__
value = self.method(instance)
^^^^^^^^^^^^^^^^^^^^^
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 96, in conda
return Conda(
^^^^^^
File "$HOME/micromamba/envs/snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 659, in __init__
self.prefix_path = self.info["conda_prefix"]
~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'conda_prefix'
I guess this might be a problem with environment variables, because my .bashrc file has been slightly set up.
The content of my .bashrc file is:
export PATH=~/local/htop/bin:$PATH
export PATH="$HOME/local/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/local/lib:$LD_LIBRARY_PATH"
# >>> mamba initialize >>>
# !! Contents within this block are managed by 'mamba init' !!
export MAMBA_EXE='$HOME/bin/micromamba';
export MAMBA_ROOT_PREFIX='$HOME/micromamba';
__mamba_setup="$("$MAMBA_EXE" shell hook --shell bash --root-prefix "$MAMBA_ROOT_PREFIX" 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__mamba_setup"
else
alias micromamba="$MAMBA_EXE" # Fallback on help from mamba activate
fi
unset __mamba_setup
# <<< mamba initialize <<<
source ~/.biostar.sh
The content of .biostar.sh is:
#
# A minimal BASH profile.
#
# Sets up the environment for the Biostar Handbook.
#
# This file is sourced by the .bashrc file.
#
# Don't edit this file directly, it may be rewritten by the install script.
#
# ls uses different flags for colors on Mac and Linux
if [ "$(uname)" == "Darwin" ]; then
alias ls='ls -hG'
else
alias ls='ls -h --color=auto'
fi
# Safe versions of the default commands.
# Ask permissions before overwriting files.
alias rm='rm -i'
alias mv='mv -i'
alias cp='cp -i'
# Extend the program search PATH and add the ~/bin and ~/edirect folders.
export PATH=~/bin:~/edirect:$PATH
# Makes the prompt more user friendly: colors, hostname, path, etc.
export PS1='\[\e]0;\w\a\]\n\[\e[32m\]\u@\h \[\e[33m\]\w\[\e[0m\]\n\$ '
# Necessary for the command line sort to work correctly.
export LC_ALL=C
# This is used on macOS to turn off zsh warning.
export BASH_SILENCE_DEPRECATION_WARNING=1
# Alias conda and mamba to micromamba
alias mamba=micromamba
alias conda=micromamba
Maybe it’s because snakemake can’t recognize micromamba?
This is my first time asking a question here, any of your suggestions are very helpful to me, thank you very much.
Looks like you've aliased mamba/conda to micromamba. I'm not sure of the specific differences, but I don't think they are fully interchangeable, at least definitely not in the eyes of Snakemake, hence the issue. You can install mamba with micromamba though: micromamba install -c conda-forge mamba