We have a private repo containing a Python project that is being built by setuptools on a CI/CD server and then the wheel artifact is being pushed back to Github as a release. This process works great, however getting the resulting wheel back into other build processes that require this as a dependancy does not seem so easy.
Our original process was using git submodules, however the wheel format is much better self contained, and the consuming project does not have to worry about relative paths, because the wheel is installed straight into site-packages.
The biggest issue has come with pulling and installing the wheel in any consuming projects. Adding the url of the wheel to the requirements.txt gets pip to try to pull the wheel, but it then fails with the following:
Could not install requirement <ProjectName>==2.0 from
https://github.com/<CompanyName>/<ProjectName>/releases/download/v2.0.0/<ProjectName>-2.0-py3-none-any.whl (from -r
requirements.txt (line 27)) because of HTTP error 404 Client Error: Not Found for url
A 404 error is also returned if following the url in a private browser window. If the url is followed on a browser already logged in to GitHub, the wheel is returned. Inspecting the request that got the wheel shows that the browser sent a cookie that let GitHub know that this wheel belonged to me, but sending cookies programatically doesn't seem like a great idea.
The following questions both touch on the same topic. The accepted answer for the first question seems not very elegent - hopefully there is a better method somewhere!
How do I download binary files of a GitHub release?
pip install wheel version from private github repo
Thanks
Stuart
There's a convenient way to authenticating the pull using a GitHub access token that will act pretty transparently across quite a few different tools that speak HTTP. This is the ~/.netrc file on most systems. Try the following:
$ touch ~/.netrc && chmod 600 ~/.netrc && cat >> ~/.netrc << 'EOF'
machine github.com
login your_githubu_sername
password your_github_api_token
EOF
Last I checked, curl, wget and git (among others) will respect this file by default for authenticating HTTP(S) requests. Though I'm not sure that this is the case in every configuration, you may be surprised at how ubiquitous the support for ~/.netrc is across different tools. This is a Unixsm, I have no idea if it is supported on MacOS or Windows, but most Linux distributions have support for it in their tools