Snakemake: opening relative Github files in Snakemake modules OUTSIDE rules

I am developing several snakemake modules that will be imported by other projects, and I'd like to draw a distinction between configuration that users can alter (config/config.yaml) and settings which should not be altered (resources/internal-settings.yaml).

I would like to pass these as parameters to rules similar to how it works with configuration files:

rule some_rule:
  params:
    protected = internal-settings["not_altered"],
    unprotected = config["may_be_altered"]

Unfortunately, I have not been able to find a way to do this.

I've tried the following in a rules.smk file outside of rules:

import yaml
from pathlib import Path

with open(Path(workflow.current_basedir) / "resources/internal-config.yaml", "r") as f:
    internal_config = yaml.safe_load(f)

But it returns the following error:

TypeError in file https://raw.githubusercontent.com/hydro/modules/hydro/workflow/rules/hydro.smk, line 6:
expected str, bytes or os.PathLike object, not GithubFile

I'd expect snakemake to just read the file relative to the module file in the GitHub repository, but it seems like this is not the case...

Solution

Found the answer with some help from the snakemake community.

To import files in a way that is "module friendly", you need to use the workflow.source_path command, which is always relative to the current file you are writing in.

So, for my question above to work, assuming you are in the main Snakefile of the workflow:

conda:
    workflow.source_path("envs/global.yaml")

import yaml
with open(workflow.source_path("resources/internal.yaml"), "r") as f:
    internal = yaml.safe_load(f)

Note: the conda: command is necessary here because I am using a non-standard python library (pyyaml) to read the file's contents. If this is exported as a module, this dependency might not be there, so you need to 'inject' it through this command. Read more about that here.