I am building a snap to test integration of a python script and a python SDK with snapcraft and there appears to be a conflict when two python 'parts' are built in the same snap.
What is the best way to build a snap with multiple python modules?
I have a simple script which imports the SDK and then prints some information. I also have the python SDK library (https://help.iotconnect.io/documentation/sdk-reference/device-sdks-flavors/download-python-sdk/) in a different folder.
I have defined the two parts, and each one can be built stand alone (snapcraft build PARTNAME), however it seems the python internals are conflicting at the next step of 'staging' them together.
tree output of structure
tree
.
├── basictest
│ ├── basictest.py
│ ├── __init__.py
│ └── setup.py
├── iotconnect-sdk-3.0.1
│ ├── iotconnect
│ │ ├── assets
│ │ │ ├── config.json
│ │ │ └── crt.txt
│ │ ├── client
│ │ │ ├── httpclient.py
│ │ │ ├── __init__.py
│ │ │ ├── mqttclient.py
│ │ │ └── offlineclient.py
│ │ ├── common
│ │ │ ├── data_evaluation.py
│ │ │ ├── infinite_timer.py
│ │ │ ├── __init__.py
│ │ │ └── rule_evaluation.py
│ │ ├── __init__.py
│ │ ├── IoTConnectSDKException.py
│ │ ├── IoTConnectSDK.py
│ │ └── __pycache__
│ │ ├── __init__.cpython-38.pyc
│ │ └── IoTConnectSDK.cpython-38.pyc
│ ├── iotconnect_sdk.egg-info
│ │ ├── dependency_links.txt
│ │ ├── not-zip-safe
│ │ ├── PKG-INFO
│ │ ├── requires.txt
│ │ ├── SOURCES.txt
│ │ └── top_level.txt
│ ├── PKG-INFO
│ ├── README.md
│ ├── setup.cfg
│ └── setup.py
└── snap
└── snapcraft.yaml
9 directories, 30 files
snapcraft.yaml
name: basictest
base: core20
version: '0.1'
summary: Test snap to verifiy integration with python SDK
description: |
Test snap to verifiy integration with python SDK
grade: devel
confinement: devmode
apps:
basictest:
command: bin/basictest
parts:
lib-basictest:
plugin: python
source: ./basictest/
after: [lib-pythonsdk]
disable-parallel: true
lib-pythonsdk:
plugin: python
source: ./iotconnect-sdk-3.0.1/
Running 'snapcraft' shows tons of errors which look like conflicts between the two 'parts' related to the internals of python.
snapcraft output
snapcraft
Launching a VM.
Skipping pull lib-pythonsdk (already ran)
Skipping pull lib-basictest (already ran)
Skipping build lib-pythonsdk (already ran)
Skipping build lib-basictest (already ran)
Failed to stage: Parts 'lib-pythonsdk' and 'lib-basictest' have the following files, but with different contents:
bin/activate
bin/activate.csh
bin/activate.fish
lib/python3.8/site-packages/_distutils_hack/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/_distutils_hack/__pycache__/override.cpython-38.pyc
lib/python3.8/site-packages/pip-21.3.1.dist-info/RECORD
lib/python3.8/site-packages/pip/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/__pycache__/__main__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/build_env.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/cache.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/configuration.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/exceptions.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/main.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/pyproject.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/self_outdated_check.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/__pycache__/wheel_builder.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/autocompletion.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/base_command.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/cmdoptions.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/command_context.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/main.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/main_parser.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/parser.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/progress_bars.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/req_command.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/spinners.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/cli/__pycache__/status_codes.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/cache.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/check.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/completion.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/configuration.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/debug.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/download.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/freeze.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/hash.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/help.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/index.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/install.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/list.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/search.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/show.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/uninstall.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/commands/__pycache__/wheel.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/distributions/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/distributions/__pycache__/base.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/distributions/__pycache__/installed.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/distributions/__pycache__/sdist.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/distributions/__pycache__/wheel.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/index/__pycache__/__init__.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/index/__pycache__/collector.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/index/__pycache__/package_finder.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/index/__pycache__/sources.cpython-38.pyc
lib/python3.8/site-packages/pip/_internal/locations/__pycache__/__init__.cpython-38.pyc
... Tons more removed
Snapcraft offers some capabilities to solve this by use of the following keywords:
- `filesets`
- `stage`
- `snap`
- `organize`
To learn more about these part keywords, run `snapcraft help plugins`.
Run the same command again with --debug to shell into the environment if you wish to introspect this failure.
Main question What is the best way to build a snap with multiple python modules?
To make the solution more portable and handle different version numbers, the sub folders can use a wildcard
lib-pythonsdk:
plugin: python
source: ./iotconnect-sdk-3.0.1/
stage:
- -lib/python3.8/site-packages/pip*
- -lib/python3.8/site-packages/wheel*
- -lib/python3.8/site-packages/setuptools/
- -lib/python3.8/site-packages/_distutils_hack
- -lib/python3.8/site-packages/pkg_resources
- -bin/activate
- -bin/activate.csh
- -bin/activate.fish
- -pyvenv.cfg