pythonlinuxpipenvdebian-bookworm

Pipenv installation fails in one directory but not in the other


I have a strange problem with Python. I'm using pipenv to manage my virtual environment. I upgraded an environment from 3.9 to 3.11 and now pipenv install fails. Error is that the hashes do not match.

As soon as I copy the Pipfile to another direcotry and try it there, it works smoothly.

I deleted the Pipfile.lock in the original directory, I moved the whole project directory to another folder, I executed pipenv update, I removed the virtual environment manually, nothing helped.

The only difference in the Pipfile.lock in the two directories is that the faulty one excludes 3.3 for six and python-dateutil:

"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3"

vs.

"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2"

Removing that exclusion doesn't help either...

My assumption is that different Python versions or pip versions are used but I would not know why, as I'm executing all commands in the same session and user.

Here's the error of pipenv install

\[pipenv.exceptions.InstallError\]: ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.  
\[pipenv.exceptions.InstallError\]: certifi==2024.8.30 from [https://www.piwheels.org/simple/certifi/certifi-2024.8.30-py3-none-any.whl#sha256=3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4](https://www.piwheels.org/simple/certifi/certifi-2024.8.30-py3-none-any.whl#sha256=3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4 "https://www.piwheels.org/simple/certifi/certifi-2024.8.30-py3-none-any.whl#sha256=3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4") (from -r /tmp/pipenv-d3ppdela-requirements/pipenv-qz_5cdgh-hashed-reqs.txt (line 2)):  
\[pipenv.exceptions.InstallError\]: Expected sha256 922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8  
\[pipenv.exceptions.InstallError\]: Expected or bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9  
\[pipenv.exceptions.InstallError\]: Got 3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4

Python 3.11 on Raspberry Pi OS bookwork 64bit.


Solution

  • First of all, don't ignore the hash mismatch — it's important to verify that the file you downloaded is the file you were expecting.

    The sha256 hashes for certifi==2024.8.30 that are public in pypi.org are

    .tar.gz bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9
    .whl    922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8
    

    see https://pypi.org/project/certifi/2024.8.30/#files

    whereas in piwheels.org the wheel is hashed

    .whl    3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4
    

    See https://piwheels.org/simple/certifi/certifi-2024.8.30-py3-none-any.whl#sha256=3dffae5ce57c3934b457066e04f14270151dd908412a601a3abb554c5acff9d4

    During the locking phase the hashes from pypi.org are whitelisted, but when installing you are sometimes trying to download certifi from piwheels.org instead, and hence the hash mismatch. So the difference between your directories isn't the python or pip version (the latter being pinned and vendored inside pipenv) but the public index you are trying to use for downloads vs locking.

    Because certifi is a pure python package (no C extensions nor alike) I can only assume that, instead of simply mirroring pipy.org, piwheels.org is repackaging the library but getting some differences in the metadata (hopefully).

    You should download both wheels from each index, unzip them, and find the differences by checking file per file and byte per byte.

    piwheels.org FAQ has a section dedicated to this question. See "I got a hash mismatch when trying to install a package from piwheels". They encourage you to open an issue in this situations.

    A workaround is to simply avoid the piwheels.org index: there is no real benefit to using it for pure python libraries.

    Another discouraged workaround is to append the piwheels hashes to your lockfile.