piprequirements.txtvirtual-environment

How to update a requirements file automatically when new packages are installed?


Keeping track of a virtual environment's requirements via pip freeze is simple.

pip freeze > requirements.txt

Currently, however, whenever a new package is added to the venv, it needs to be added to the requirements file manually. To do so, I usually just run the freeze command again and pipe it into the requirements file, but sometimes I forget to run this command, and this can be troublesome especially in repositories across different locations whenever I have to remember which packages I need to install!

Whenever a new package is installed in a virtual environment, is there any way to automatically update a requirements.txt file automatically to include this new package?


Solution

  • When using just plain pip, there is currently no way to make it automatically generate or update a requirements.txt file. It is still mostly a manual process using pip freeze > requirements.txt (see related How to create a requirements.txt for other options).

    If the purpose is to make sure the installed packages are tracked or registered properly (i.e. tracked in version control for a repository), then you will have to use other tools that "wrap" around pip's functionality.

    You have two options.

    Option 1: Use a package/dependency manager

    There are a number of Python package/dependency managers that combines "install package" with "record installed packages somewhere". See the Managing Application Dependencies section of the Python docs. Some common examples are:

    Option 2: git pre-commit hook

    This solution isn't going to happen "during installation of the package", but if the purpose is to make sure your tracked "requirements.txt" is synchronized with your virtual environment, then you can add a git pre-commit hook that:

    1. Generates a separate requirements_check.txt file
    2. Compares requirements_check.txt to your requirements.txt
    3. Aborts your commit if there are differences

    Example .git/hooks/pre-commit:

    #!/usr/local/bin/bash
    
    pip freeze > requirements_check.txt
    cmp --silent requirements_check.txt requirements.txt
    
    if [ $? -gt 0 ]; 
    then
      echo "There are packages in the env not in requirements.txt"
      echo "Aborting commit"
      rm requirements_check.txt
      exit 1
    fi
    
    rm requirements_check.txt
    exit 0
    

    Output:

    $ git status
    ...
    nothing to commit, working tree clean
    
    $ pip install pydantic
    
    $ git add .
    $ git commit
    The output of pip freeze is different from requirements.txt
    Aborting commit
    
    $ pip freeze > requirements.txt
    $ git add .
    $ git commit -m "Update requirements.txt"
    [master 313b685] Update requirements.txt
     1 file changed, 1 insertion(+)