pythondependenciescondasnakemake

Problems with MultiQC in a conda environment within snakemake pipeline

This is the problematic rule:

rule multiqc_trimmed:
    input:
        expand("results/read_quality/QC_trimmed/fastqc/{sample}_{pr}.trimmed_fastqc.zip", sample=config['samples'], pr=MULTIFR) 
    output:
        "results/read_quality/QC_trimmed/qcreport_trimmed.html"
    params:
        outdir="results/read_quality/QC_trimmed"
    threads: 1
    log:
        "workflow/logs/multiqc_trimmed/multiqc.log"
    benchmark:
        "workflow/benchmarks/multiqc_trimmed/multiqc.tsv"
    conda:
        "../envs/qc.yaml"
    shell:
        """
        FILENAME=$(basename {output})
        multiqc {params.outdir} -n $FILENAME -o {params.outdir} &>> {log}
        """

This is qc.yaml

name: cel-qc
channels:
  - bioconda
  - conda-forge
dependencies:
  - fastqc =0.12.1
  - multiqc =1.17

This is the log file:

  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/bin/multiqc", line 6, in <module>
    from multiqc.__main__ import run_multiqc
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/multiqc/__init__.py", line 16, in <module>
    from .multiqc import run
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/multiqc/multiqc.py", line 25, in <module>
    import rich_click as click
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/rich_click/__init__.py", line 16, in <module>
    from rich_click.rich_command import RichCommand
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/rich_click/rich_command.py", line 5, in <module>
    from rich_click.rich_click import rich_abort_error, rich_format_error, rich_format_help
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/rich_click/rich_click.py", line 5, in <module>
    import rich.markdown
  File "/home/cel-rnaseq/.snakemake/conda/5cfa1cb4cfd5e47de4c18a25342a7d05_/lib/python3.6/site-packages/rich/markdown.py", line 1
    from __future__ import annotations
    ^
SyntaxError: future feature annotations is not defined

I thought it was a python error so I added python 3.10 / python 3.12 to the environment, changed multiqc to 1.27.1 (latest stable release) and I get some variation of errors like: [1]

The following packages are incompatible
├─ multiqc =1.17 * is installable and it requires
│  └─ requests =* * with the potential options
│     ├─ requests [2.10.0|2.11.0|...|2.9.2] would require
│     │  └─ python =2.7 *, which can be installed;
│     ├─ requests [2.10.0|2.11.0|...|2.9.2] would require
│     │  └─ python =3.4 *, which can be installed;
│     ├─ requests [2.10.0|2.11.0|...|2.9.2] would require
│     │  └─ python =3.5 *, which can be installed;
│     ├─ requests [2.12.4|2.12.5|2.13.0] would require
│     │  └─ python =3.6 *, which can be installed;
│     ├─ requests [2.17.3|2.18.1|2.18.2|2.18.3|2.18.4] would require
│     │  └─ urllib3 >=1.21.1,<1.22 *, which conflicts with any installable versions previously reported;
│     ├─ requests 2.18.4 would require
│     │  └─ urllib3 >=1.21.1,<1.23 *, which conflicts with any installable versions previously reported;
│     ├─ requests [2.19.0|2.19.1|2.20.0|2.20.1] would require
│     │  └─ urllib3 >=1.21.1,<1.24 *, which conflicts with any installable versions previously reported;
│     ├─ requests [2.21.0|2.22.0] would require
│     │  └─ urllib3 >=1.21.1,<1.25 *, which conflicts with any installable versions previously reported;
│     ├─ requests [2.22.0|2.23.0|2.24.0|2.25.0] would require
│     │  └─ urllib3 >=1.21.1,(<1.26,(!=1.25.0,!=1.25.1)) *, which conflicts with any installable versions previously reported;
│     ├─ requests [2.25.1|2.26.0|...|2.29.0] would require
│     │  └─ urllib3 >=1.21.1,<1.27 *, which conflicts with any installable versions previously reported;
│     └─ requests [2.31.0|2.32.1|2.32.2|2.32.3] would require
│        └─ urllib3 >=1.21.1,<3 *, which conflicts with any installable versions previously reported;
└─ python =3.10 * is not installable because it conflicts with any installable versions previously reported.

and [2] (tried setting up the environment outside snakemake)

error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ multiqc =1.27.1 * is not installable because it requires
       └─ tiktoken =* *, which requires
          └─ regex >=2022.1.18 *, which conflicts with any installable versions previously reported.
critical libmamba Could not solve for environment specs

and [3]

error    libmamba Could not solve for environment specs
    The following packages are incompatible
    ├─ multiqc =1.27.1 * is not installable because it requires
    │  └─ pillow >=10.2.0 * but there are no viable options
    │     ├─ pillow [10.2.0|10.3.0|10.4.0] would require
    │     │  └─ libtiff >=4.6.0,<4.8.0a0 * but there are no viable options
    │     │     ├─ libtiff [4.6.0|4.7.0] would require
    │     │     │  └─ libdeflate >=1.21,<1.22.0a0 *, which conflicts with any installable versions previously reported;
    │     │     ├─ libtiff 4.7.0 would require
    │     │     │  └─ libdeflate >=1.22,<1.23.0a0 *, which conflicts with any installable versions previously reported;
    │     │     ├─ libtiff 4.7.0 would require
    │     │     │  └─ libdeflate >=1.23,<1.24.0a0 *, which conflicts with any installable versions previously reported;
    │     │     ├─ libtiff 4.6.0 would require
    │     │     │  └─ libdeflate >=1.20,<1.21.0a0 *, which conflicts with any installable versions previously reported;
    │     │     ├─ libtiff 4.6.0 would require
    │     │     │  └─ libdeflate >=1.19,<1.20.0a0 *, which conflicts with any installable versions previously reported;
    │     │     └─ libtiff 4.6.0 would require
    │     │        └─ libdeflate >=1.18,<1.19.0a0 *, which conflicts with any installable versions previously reported;
    │     └─ pillow [11.0.0|11.1.0] would require
    │        └─ libtiff >=4.7.0,<4.8.0a0 *, which cannot be installed (as previously explained);
    └─ regex =2024.11.6 * is not installable because it conflicts with any installable versions previously reported.
critical libmamba Could not solve for environment specs

When I manually install multiqc when inside the conda environment, it works fine. As soon as I use a config file, it breaks in some variation of these errors. Should I try to define each package that gives errors in the config file? I partially tried it but got other dependency errors (see error [3] where I added regex=2024.11.6).

Solution

  • It worked once I flipped the conda channels around, ended up with this config file:

    name: cel-qc
channels:
  - conda-forge
  - bioconda
dependencies:
  - python =3.10
  - fastqc =0.12.1
  - multiqc =1.27.1