I am switching a project that currently uses pipenv to poetry as a test to see what the differences are. The project is a simple, redistributable, Django app. It supports Python 3.6-8, and Django 2.2 and 3.0. I have a tox.ini file that covers all combinations of Python and Django thus:
[tox]
envlist = py{36,37,38}-django{22,30}
[testenv]
whitelist_externals = poetry
skip_install = true
deps =
django22: Django==2.2
django30: Django==3.0
commands =
poetry install -vvv
poetry run pytest --cov=my_app tests/
poetry run coverage report -m
The problem that I am having (which does not exist in the pipenv world) is that the poetry install statement will always overwrite whatever is in the deps section with whatever is in the poetry.lock file (which will be auto-generated if it does not exist). This means that the test matrix will never test against Django 2.2 - as each tox virtualenv gets Django 3.0 installed by default.
I don't understand how this is supposed to work - should installing dependencies using poetry respect the existing environment into which it is being installed, or not?
How do I set up a multi-version tox (or Travis) test matrix, with poetry as the dependency manager?
My pyproject.toml defines Python / Django versions as:
[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"
The generated poetry.lock file (not committed) has this Django version information:
[[package]]
category = "main"
description = "A high-level Python Web framework that encourages rapid development and clean, pragmatic design."
name = "django"
optional = false
python-versions = ">=3.6"
version = "3.0"
This is the result when I delete the lock file, and recreate the tox environment. As you can see, tox installs Django==2.2 as a dependency in the virtualenv, but poetry then updates this to 3.0 when it installs.
my-app$ tox -r -e py36-django22
py36-django22 recreate: .tox/py36-django22
py36-django22 installdeps: Django==2.2
py36-django22 installed: Django==2.2,my-app==0.1.0,pytz==2019.3,sqlparse==0.3.0
py36-django22 run-test: commands[0] | poetry install -vvv
Using virtualenv: .tox/py36-django22
Updating dependencies
Resolving dependencies...
1: derived: django (^2.2 || ^3.0)
...
PyPI: 10 packages found for django >=2.2,<4.0
...
1: Version solving took 3.330 seconds.
1: Tried 1 solutions.
Writing lock file
Package operations: 52 installs, 1 update, 0 removals, 3 skipped
- ...
- Updating django (2.2 -> 3.0)
- ...
I need a solution that runs the poetry install, respecting existing package installs. i.e. if pyproject.toml states Django = "^2.2 || ^3.0", and 2.2 is already installed, then pin to that version - don't attempt to upgrade.
Haven't thoroughly tested it, but I believe something like this should work:
[tox]
envlist = py{36,37,38}-django{22,30}
isolated_build = True
[testenv]
deps =
django22: Django==2.2
django30: Django==3.0
# plus the dev dependencies
pytest
coverage
commands =
pytest --cov=my_app tests/
coverage report -m
See the "poetry" section in the "packaging" chapter of the tox documentation.
In order to avoid the repetition of the dev dependencies, one could try the following variation based on the extras feature:
tox.ini
[tox]
# ...
[testenv]
# ...
deps =
django22: Django==2.2
django30: Django==3.0
extras =
test
pyproject.toml
[tool.poetry]
# ...
[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"
#
pytest = { version = "^5.2", optional = true }
[tool.poetry.extras]
test = ["pytest"]
[build-system]
# ...
Nowadays there are tox plug-ins that try to make for a better integration with poetry-based projects:
tox-poetry-installertox-poetry-dev-dependencies (disclaimer: my project)tox-poetry