I wish to use some C and CUDA code in my Python package (which I then call using ctypes). Because of the CUDA, it doesn't seem to be easy to use the traditional approach of a setuptools Extension, so I instead pre-compile the code to shared libraries and then wish to include them in a Wheel. When I build the Wheel it includes the shared libraries, but it still gives the output Wheel a "none-any" extension, indicating that it is pure Python and platform independent, which is not correct. cibuildwheel then refuses to run auditwheel on it because of this. I have read posts about forcing Wheels to be labelled as platform dependent, but I imagine that I shouldn't have to force it and that I am instead doing something wrong.
Directory structure:
pyproject.toml:
[build-system]
requires = ["setuptools>=42"]
build-backend = "setuptools.build_meta"
[tool.cibuildwheel]
build = "cp36-*"
skip = ["pp*", "*686"]
manylinux-x86_64-image = "manylinux2014"
[tool.cibuildwheel.linux]
before-all = "bash {project}/src/mypackage/before_all.sh"
setup.cfg:
[metadata]
name = mypackage
[options]
package_dir =
= src
include_package_data = True
zip_safe = False
packages = find:
python_requires = >=3.6
[options.packages.find]
where = src
[options.package_data]
mypackage =
*.so
Am I missing something that is causing the packager to not detect that the Wheel is not pure Python and platform independent, or is forcing the packager to label it otherwise (such as by creating an empty Extension) the normal approach?
I was able to get it to work using the method referred to by @phd, and I also got it to work using Hatchling instead of Setuptools as the backend. In both cases auditwheel is successful, finding the shared libraries and including their dependencies.
Both methods feel fragile, however. I have come to the realisation that Wheels are intended for Python extensions. I have no need for my compiled code to have any knowledge of Python, and while I could wrap calls to my compiled code using the likes of CFFI or Cython, I don't see an advantage of it in my case and it would introduce a dependence on the Python ABI.
I have thus decided to only provide a source distribution (no Wheels), including the shared libraries for all platforms in the source distribution and loading the correct one for the platform at runtime.
I should note that there are extensions of Setuptools that might also meet my needs, such as setuptools-cuda-cpp to enable Setuptools to compile CUDA, and setuptools_dso to build non-Python shared libraries for inclusion in Wheels with Setuptools.
In case it is useful to someone else, to get Setuptools and Hatchling to produce Wheels that declare themselves to only require Python 3 (not a particular release of CPython), have no Python ABI dependence, and be for the Linux x86_64 platform, I had to make the below changes.
Setuptools:
setup.py:
from setuptools import setup, Distribution
try:
from wheel.bdist_wheel import bdist_wheel as _bdist_wheel
class MyWheel(_bdist_wheel):
def finalize_options(self):
_bdist_wheel.finalize_options(self)
self.root_is_pure = False
def get_tag(self):
python, abi, plat = _bdist_wheel.get_tag(self)
python, abi = 'py3', 'none'
return python, abi, plat
class MyDistribution(Distribution):
def __init__(self, *attrs):
Distribution.__init__(self, *attrs)
self.cmdclass['bdist_wheel'] = MyWheel
def is_pure(self):
return False
def has_ext_modules(self):
return True
except ImportError:
class MyDistribution(Distribution):
def is_pure(self):
return False
def has_ext_modules(self):
return True
setup(
distclass=MyDistribution
)
Hatchling:
hatch_build.py:
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
class CustomHook(BuildHookInterface):
def initialize(self, version, build_data):
build_data['pure_python'] = False
build_data['tag'] = 'py3-none-linux_x86_64'
and then, in addition to the other changes to pyproject.toml needed to use Hatchling, the line
[tool.hatch.build.targets.wheel.hooks.custom]
needs to be included to run the hook in the above hatch_build.py.