pythonsetuptoolspython-wheeldistribute

How to Package and Publish a Script That Calls Other Scripts as a Pip Wheel


I'm trying to create wheel for scripts generated with bazel py_binary rule, and encountered some problem.

I try to give a minor example of the problem. I have a directory like this:

.
├── MANIFEST.in
├── bin
│   ├── __init__.py
│   ├── mytest
│   └── mytest.lib
│       ├── used1
│       └── used2
└── setup.py

where bin/mytest.lib contains many scripts, and bin/mytest is the entry of this package, calling all the scripts in bin/mytest.lib. It could look like:

#!/bin/bash

bash $0.lib/used1
bash $0.lib/used2

I write a setup.py to create a wheel:

import setuptools

setuptools.setup(
    name="mytest",
    version="0.0.1",
    maintainer="",
    maintainer_email="",
    packages=setuptools.find_packages(),
    install_requires=[],
    scripts=["bin/mytest"],
    include_package_data=True,
)

and MANIFEST.in tries to get all files in bin:

graft bin

Then, I python setup.py bdist_wheel and pip install dist/mytest-0.0.1-py3-none-any.whl.

I tried to use mytest but failed as MY_ENV/bin/mytest.lib/used1 doesn't exist. (Here MY_ENV == /home/dev/conda_dev/devenv/Linux/envs/devenv-3.8-c)

There are 2 major problems:

  1. mytest.lib is in MY_ENV/lib/python3.8/site-packages/bin rather than MY_ENV/lib/python3.8/site-packages/mytest-0.0.1.dist-info/bin
  2. mytest actually expects mytest.lib be in MY_ENV/bin rather than MY_ENV/lib/python3.8/site-packages/mytest-0.0.1.dist-info/bin or other directories.

If I cannot modify mytest or mytest.lib as they are generated by bazel (but I can add something if needed), how can I make mytest available?


Solution

  • I'm inspired from flask8 and create such a workaround:

    I change directory to

    .
    ├── mytest
    │   ├── entry.py
    │   ├── __init__.py
    │   ├── mytest
    │   ├── mytest.lib
    │   │   ├── used1
    │   │   └── used2
    │   └── tmp.md
    └── setup.py
    

    so that I'll get MY_ENV/lib/python3.8/site-packages/mytest.

    I put mytest.py as entry_points, my setup.py currently looks like:

    import setuptools
    
    __version__ = "0.0.1"
    
    setuptools.setup(
        name="mytest",
        version=__version__,
        maintainer="",
        maintainer_email="",
        packages=["mytest"],
        install_requires=[],
        entry_points={
            'console_scripts': ['mytest = mytest.entry:entry']
        },
        package_data={'mytest': ['**/*']},
    )
    

    and in entry, I call mytest script:

    import os
    import sys
    import subprocess
    
    def entry():
        argv = sys.argv[1:]
        dir = os.path.dirname(os.path.abspath(__file__))
        script = os.path.join(dir, "mytest")
        subprocess.run([script, *argv])
    

    It's currently working well for me, but I don't know whether this is a good solution. I'm looking forward to any better solutions :)