pythonpipdependency-managementrelease-managementnightly-build

Bleeding Edge/Nightly Build dependency management in Python. Strategies and best practices?



The situation


You have 2 software products in development, a library which presents an API and a GUI tool that exposes the library for end users. Additionally, you expect a lot of technical staff at your place to use the library as a building block for all sorts of relevant custom code, tools and assets.

Both software products (library and GUI tool) are actively in development and influence each other. For both you want the most easy way of distribution and devenv setup, using pip:

pip install gui_tool
or
pip install library

Gui Tool (use case 1)


The installation of the GUI tool happens via pip and the dependencies are noted with a hard version number within the setup.py. Your library is one of those dependencies:

...
install_requires = ['library==1.2, package_x==0.3, package_y==0.6'],
...

The installation procedure consists of installing the tools and resolving the dependencies into a fresh virtualenv. Because every dependency version is hardwired, you control a stable and consistent installation. Nightly builds/bleeding edge dependencies can be controlled by devs through manually updating library to newer versions:

pip install --upgrade library  # get the latest nightly build/hotfix release on your own

Custom code/tools (use case 2)


As mentioned, a lot of code and custom tools might be built using the API that your library provides. Everybody willing to use it should install/update it through pip easily with the oneliner from the top.


Problem


Fellow GUI tool developers should be able to pull in nightly builds/hotfix releases of the library dependency with pip. Other staff, using library as a building block somewhere, should always get the latest stable version using pip. You want to remain one unique release procedure for library providing stable versions, bleeding edge and hotfix releases through X.Y.Z versioning.

There are a few possible solutions to this, like:


However, none of these seem particularly elegant, so I am interested in your solutions, ideas or best practices for stable/bleeding edge/nightly build dependency management for Python?


Solution

  • Managing a "bleeding edge" version in pip can be achieved by using the --pre flag of pip

    From pip install --help:

    --pre Include pre-release and development versions. By default, pip only finds stable versions.

    You should add classifiers to your project, in particular set your stable release to Development Status :: 5 - Production/Stable and your "bleeding edge releases" to anything below 5.

    This is how major python packages manage their alphas, e.g.: the django project with currently 1.9a in alpha and 1.8.5 in stable.

    To upgrade to the latest version flagged as Development Status :: 3 - Alpha :

    pip install --pre --upgrade library 
    

    Users using the library as building blocks need not now about the alpha releases and will go with a regular pip install library which will install the latest stable flagged version.