How to Package and Publish a Script That Calls Other Scripts as a Pip Wheel

I'm trying to create wheel for scripts generated with bazel py_binary rule, and encountered some problem.

I try to give a minor example of the problem. I have a directory like this:

.
├── MANIFEST.in
├── bin
│   ├── __init__.py
│   ├── mytest
│   └── mytest.lib
│       ├── used1
│       └── used2
└── setup.py

where bin/mytest.lib contains many scripts, and bin/mytest is the entry of this package, calling all the scripts in bin/mytest.lib. It could look like:

#!/bin/bash

bash $0.lib/used1
bash $0.lib/used2

I write a setup.py to create a wheel:

import setuptools

setuptools.setup(
    name="mytest",
    version="0.0.1",
    maintainer="",
    maintainer_email="",
    packages=setuptools.find_packages(),
    install_requires=[],
    scripts=["bin/mytest"],
    include_package_data=True,
)

and MANIFEST.in tries to get all files in bin:

graft bin

Then, I python setup.py bdist_wheel and pip install dist/mytest-0.0.1-py3-none-any.whl.

I tried to use mytest but failed as MY_ENV/bin/mytest.lib/used1 doesn't exist. (Here MY_ENV == /home/dev/conda_dev/devenv/Linux/envs/devenv-3.8-c)

There are 2 major problems:

1. mytest.lib is in MY_ENV/lib/python3.8/site-packages/bin rather than MY_ENV/lib/python3.8/site-packages/mytest-0.0.1.dist-info/bin
2. mytest actually expects mytest.lib be in MY_ENV/bin rather than MY_ENV/lib/python3.8/site-packages/mytest-0.0.1.dist-info/bin or other directories.

If I cannot modify mytest or mytest.lib as they are generated by bazel (but I can add something if needed), how can I make mytest available?

I'm inspired from flask8 and create such a workaround:

I change directory to

.
├── mytest
│   ├── entry.py
│   ├── __init__.py
│   ├── mytest
│   ├── mytest.lib
│   │   ├── used1
│   │   └── used2
│   └── tmp.md
└── setup.py

so that I'll get MY_ENV/lib/python3.8/site-packages/mytest.

I put mytest.py as entry_points, my setup.py currently looks like:

import setuptools

__version__ = "0.0.1"

setuptools.setup(
    name="mytest",
    version=__version__,
    maintainer="",
    maintainer_email="",
    packages=["mytest"],
    install_requires=[],
    entry_points={
        'console_scripts': ['mytest = mytest.entry:entry']
    },
    package_data={'mytest': ['**/*']},
)

and in entry, I call mytest script:

import os
import sys
import subprocess

def entry():
    argv = sys.argv[1:]
    dir = os.path.dirname(os.path.abspath(__file__))
    script = os.path.join(dir, "mytest")
    subprocess.run([script, *argv])

It's currently working well for me, but I don't know whether this is a good solution. I'm looking forward to any better solutions :)