How could the content of pip install and PyPI download be different? is pip broken?

Today I tried to install the latest pymavlink package:

pip install pymavlink
Collecting pymavlink
  Using cached pymavlink-2.4.41-py3-none-any.whl.metadata (6.2 kB)
Requirement already satisfied: future in /home/shared/conda/envs/swarm/lib/python3.10/site-packages (from pymavlink) (1.0.0)
Requirement already satisfied: lxml in /home/shared/conda/envs/swarm/lib/python3.10/site-packages (from pymavlink) (5.3.0)
Using cached pymavlink-2.4.41-py3-none-any.whl (11.6 MB)
Installing collected packages: pymavlink
Successfully installed pymavlink-2.4.41

after installation I found that the source directory is surprisingly small, it only contains the following files:

/dialects /generator /message_definitions /__pycache__ /CSVReader.py /DFReader.py /fgFDM.py /__init__.py /mavexpression.py /mavextra.py /mavparm.py /mavtestgen.py /mavutil.py /mavwp.py /quaternion.py /rotmat.py /setup.py

But when I download it directly from PyPI, it contains the following files:

/dialects /generator /message_definitions /pymavlink.egg-info /tests /tools /COPYING /CSVReader.py /DFReader.py /fgFDM.py /__init__.py /mavexpression.py /mavextra.py /mavparm.py /mavtestgen.py /mavutil.py /mavwp.py /PKG-INFO /pyproject.toml /quaternion.py /README.md /rotmat.py /setup.cfg /setup.py

the pip version has several directory missing. How could this happen? How to prevent it?

UPDATE 1: fearing that it may be caused by defective publishing, it has also been posted on pymavlink issue tracker:

https://github.com/ArduPilot/pymavlink/issues/969

Solution

Looking at the two list of files, the differences seem reasonable for the two different types of install.

The "source" install has additional files and directories, but these are to reflect the "source" repository that are not required for end-user installs.

Looking at your git issue you raised, it appears you think you are missing tools/mavgen.py. Looking in the setup.py file, it has instructions to install that file in $VIRTUAL_ENV/bin. And if I look in that directory then I can see the file. The $VIRTUAL_ENV/bin directory gets added to $PATH when the Python Virtual Environment is activated so I did the following test:

(venv) $ mavgen.py -h
usage: mavgen.py [-h] [-o OUTPUT]
                 [--lang {Ada,C,CS,JavaScript,JavaScript_Stable,JavaScript_NextGen,TypeScript,Python2,Python3,Python,Lua,WLua,ObjC,Swift,Java,C++11}]
                 [--wire-protocol {0.9,1.0,2.0}] [--no-validate]
                 [--error-limit ERROR_LIMIT] [--strict-units] [--exit-code]
                 XML [XML ...]

This tool generate implementations from MAVLink message definitions

positional arguments:
  XML                   MAVLink definitions

options:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        output directory.
  --lang {Ada,C,CS,JavaScript,JavaScript_Stable,JavaScript_NextGen,TypeScript,Python2,Python3,Python,Lua,WLua,ObjC,Swift,Java,C++11}
                        language of generated code [default: Python]
  --wire-protocol {0.9,1.0,2.0}
                        MAVLink protocol version. [default: 1.0]
  --no-validate         Do not perform XML validation. Can speed up code
                        generation if XML files are known to be correct.
  --error-limit ERROR_LIMIT
                        maximum number of validation errors to display
  --strict-units        Perform validation of units attributes.
  --exit-code           Return an error code if generation fails.

This all seems to be working as expected.

I would expect your correct command line to be:

mavgen.py --lang="$i" --wire-protocol=2.0 ...

rather than

python -m pymavlink.tools.mavgen --lang="$i" --wire-protocol=2.0 ...