I am developing several snakemake
modules that will be imported by other projects, and I'd like to draw a distinction between configuration that users can alter (config/config.yaml
) and settings which should not be altered (resources/internal-settings.yaml
).
I would like to pass these as parameters to rules
similar to how it works with configuration files:
rule some_rule:
params:
protected = internal-settings["not_altered"],
unprotected = config["may_be_altered"]
Unfortunately, I have not been able to find a way to do this.
I've tried the following in a rules.smk
file outside of rules:
import yaml
from pathlib import Path
with open(Path(workflow.current_basedir) / "resources/internal-config.yaml", "r") as f:
internal_config = yaml.safe_load(f)
But it returns the following error:
TypeError in file https://raw.githubusercontent.com/hydro/modules/hydro/workflow/rules/hydro.smk, line 6:
expected str, bytes or os.PathLike object, not GithubFile
I'd expect snakemake
to just read the file relative to the module file in the GitHub repository, but it seems like this is not the case...
Found the answer with some help from the snakemake
community.
To import files in a way that is "module friendly", you need to use the workflow.source_path
command, which is always relative to the current file you are writing in.
So, for my question above to work, assuming you are in the main Snakefile
of the workflow:
conda:
workflow.source_path("envs/global.yaml")
import yaml
with open(workflow.source_path("resources/internal.yaml"), "r") as f:
internal = yaml.safe_load(f)
Note: the conda:
command is necessary here because I am using a non-standard python library (pyyaml
) to read the file's contents. If this is exported as a module, this dependency might not be there, so you need to 'inject' it through this command. Read more about that here.