[SOLVED] Why does my pip access two site-packages?

Why does my pip access two site-packages?

I have a bit of confusion with pip and multiple python installations. When running python -m pip install pb_tool I get console output:

Requirement already satisfied: pb_tool in c:\osgeo4w\apps\python39\lib\site-packages (3.1.0)
Requirement already satisfied: colorama in c:\users\hbh1\appdata\roaming\python\python39\site-packages (from pb_tool) (0.4.6)
Requirement already satisfied: Sphinx in c:\users\hbh1\appdata\roaming\python\python39\site-packages (from pb_tool) (6.1.1)
Requirement already satisfied: Click in c:\osgeo4w\apps\python39\lib\site-packages (from pb_tool) (7.1.2)
...

I am wondering, why there are mixed site-packages paths, some in c:\osgeo4w\apps\ and some in c:\users\hbh1\appdata\...?

I installed pb_tool with the OSGeo4W python, I'd expect to find it and its requirements found/installed in c:\osgeo4w\..., not (even partially?!) in the c:\users\hbh1\appdata\..., especially when running pip with python -m.

To elaborate: This is not necessarily a problem, but I'd like to understand, why and also if/how I can circumvent this behavior. It caused me some confusion as to which python installation has which modules installed, and I'd like to keep things separate and an overview over where I installed what.

Some time back I ran pip install pb_tool in my dev shell and couldn't run pb_tool afterwards, despite the successful install. I assume, the problem is, that I didn't have c:\users\hbh1\appdata\roaming\python\python39\site-packages on the PATH in that current environment. But somehow pip knew it, installed pb_tool there and python didn't know about it (I didn't add it, since I'd like a "clean and separated" dev enviroment with it's own python packages)... I carefully checked PATH, my python/pip versions and which is which (cleaning PATH, using where pip/where python and py -0b to check the windows python launcher as well). My setup basically is:

  # add to PATH depending on the version I use
  C:\Apps\Python39\
  C:\OSGeo4W\apps\Python39  # respectively C:\OSGeo4W\bin

  # and their corresponding script dirs
  C:\Apps\Python39\Scripts
  C:\Users\hbh1\AppData\Roaming\Python\Python39\Scripts
  C:\OSGeo4W\apps\Python39\Scripts

  # and if relevant: Windows Python Launcher listing these (py -0p), where I only use the first (the second one is not on PATH):
  -3.9-64        C:\Apps\Python39\python.exe *
  -2.7-32        C:\Apps\ArcGIS\Python27\ArcGIS10.8\python.exe

C:\OSGeo4W\ is a dev environment for me and I use a "clean" shell for command line tools I use with it (meaning that I don't use the system PATH, but start with a .bat in which I clean the PATH and only add what I specifically need, plus a few general sytem paths).

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
set path=%OSGEO4W_ROOT%\bin;%WINDIR%\system32;%WINDIR%;%WINDIR%\system32\WBem
path %PATH%;%OSGEO4W_ROOT%\apps\Python39\Scripts
path %PATH%;%OSGEO4W_ROOT%\apps\qgis-ltr\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass78\lib
path %PATH%;%OSGEO4W_ROOT%\apps\Qt5\bin

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis-ltr\python
set PYTHONHOME=%OSGEO4W_ROOT%\apps\Python39

set PATH=C:\Program Files\Git\bin;%PATH%

cmd.exe

I am still puzzled, why in this environment, pip install would put anything in the c:\users\hbh1\appdata\roaming\python\python39\site-packages which is "normally" used by my C:\Apps\Python39\ installation.

Solution

Try setting PYTHONNOUSERSITE so that Python won't add the user site-packages directory to sys.path. This can be made permanent in OSGeo4W by adding this line to python3.bat:

SET PYTHONNOUSERSITE=1

the location of this configuration file may depend on how OSGeo4W was installed, mine is in C:\OSGeo4W\etc\ini\