condaexebinaryfilesconda-build

How to create conda package for a tool

I am new to creating Conda packages. Here is my problem:

I have a tool that has just one binary executable program. There is no other dependencies. I tried generating a Conda package for the same as follows:

  1. I made a meta.yaml, which contains:
package:
  name: XXXXX
  version: 1.0

source:
  path: $HOME/bin/XXXXX/

build:
  script:
    - chmod -R 777 $HOME/bin/XXXXX/YYY
    - export PATH=$HOME/bin/XXXXX/:$PATH

about:
  license: open-source
  1. I build the Conda package using
conda build XXXXX

Step 2 generates a file like:

/conda-bld/linux-64/XXXXX-1.0-0.tar.bz2
  1. When I ran
conda install --offline  /conda-bld/linux-64/XXXXX-1.0-0.tar.bz2

it is showing up on conda list as

XXXXX    1.0    0    file:///$HOME/xxxxxxx/xxxxx/xxxxx/conda-bld

But when try fetching XXXXX using clicking tab nothing is appearing.

The aim of creating this package is that after Conda installation, YYY should be accessible inside the Conda environment.

Any idea how to get this working properly?

Solution

  • There are a few issues.

    Copy to $PREFIX/bin

    The main issue here is trying to set a shell environment variable. This won't live outside the process building the package. Rather, the way Conda packages is work is by copy files into paths relative to the environment prefix. Hence, what you really want to do is copy the binary to $PREFIX/bin and then add the executable permissions. Something like,

    build:
  number: 0
  script:
    - "mkdir -p $PREFIX/bin"
    - "cp PromPredict_mulseq $PREFIX/bin/PromPredict_mulseq"
    - "chmod u+x $PREFIX/bin/PromPredict_mulseq"

    Don't rely on $HOME

    The other issue is probably how the source is being specified. I'm not sure the $HOME variable will get resolved correctly. I believe it is more reliable to have some relative path (e.g., ../PromPredict_mulseq), and better to have a tarball to point at. You may need to debug this a bit to ensure the binary is getting located. It can help to run conda build with a --dirty flag and then inspect the work folder under the conda-bld directory.

    Install by name with --use-local flag

    Finally, rather than specifying the full path to the built file, one should prefer specifying by name and then including a --use-local flag, e.g.,

    conda install --use-local prompredict

    Additional Discussion

    It might be helpful to inspect a concrete example package that copies binaries. Consider looking at the meta.yaml and build.sh for the STAR recipe in the Bioconda repository. It simply copies the binary from the GitHub tarball upstream.

    Generally, I don't recommend writing recipes that copy binaries, but I can't deny it's the path of least resistance. Instead, I would prefer compiling the software as part of the recipe. This let's the binary use dynamic linked libraries rather than being a static build, and this is one of the most remarkable strengths of Conda.