testingcode-coveragecoverage.py

How do I selectively hide no-data-collected when using coverage.py?


I have a test suite that runs a bunch of system tests on an API. Some system tests interact with the API by using Flask's test_client, whereas a few specific tests interact with the API by making HTTP queries to a Gunicorn running the API.

When I use coverage, as soon as it hits those special tests that use Gunicorn, a warning is shown:

CoverageWarning: No data was collected. (no-data-collected)

The warning makes perfect sense—coverage cannot track the actual coverage in tests when the tested API runs through Gunicorn, instead of being called directly. However, it's annoying to see this warning at every test run.

How do I hide it for those specific tests, while ensuring it will be shown if the coverage for other tests starts misbehaving?


A few additional details:

All tests (both the ones that rely on Flask's test_client and the ones that do HTTP queries to Gunicorn) are run in one go with the command:

coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast

The contents of the aforementioned coverage-options file are:

[run]
branch = True
concurrency = multiprocessing
omit =
    /usr/*/dist-packages/*
    */.local/lib/*/site-packages/*
    tests/*

The tests/ directory contains several sub-directories:

The output is:

(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...................................................................................................................................../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
.../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
.......
----------------------------------------------------------------------
Ran 143 tests in 45.853s

OK

As tests are ran in alphabetical order, the first one hundred and thirty something dots match the tests in tests/api/. The last seven correspond to the unit tests. The warnings are shown once the very first test from tests/system/ gets executed, and then once again when all four are executed.

By comparison, if I remove tests/system/ directory, this is what I get:

(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...........................................................................................................................................
----------------------------------------------------------------------
Ran 139 tests in 38.022s

OK

Solution

  • Trying to create a minimal reproducible example, it appears that the warning is emitted every time there is a multiprocessing.Process in the test, and more precisely when this process terminates.

    Here are the steps to get the warning.

    mkdir /tmp/ndc
    cd /tmp/ndc
    python3 -m venv .venv
    source .venv/bin/activate
    pip install coverage
    

    Create the following tree in /tmp/ndc:

    tests/
      __init__.py (empty)
      test_demo.py ①
    coverage-options ②
    

    Contents of /tmp/ndc/tests/test_demo.py ①:

    import multiprocessing
    import time
    import unittest
    
    class DemoTests(unittest.TestCase):
        def test_with_multiprocessing(self):
            p = multiprocessing.Process(target=t)
            p.start()
            p.join()
            self.assertTrue(2 * 2 == 4)
    
    
    def t():
        time.sleep(0.1)
    

    Contents of /tmp/ndc/coverage-options ②:

    [run]
    branch = True
    concurrency = multiprocessing
    omit =
        /usr/*/dist-packages/*
        */.local/lib/*/site-packages/*
        tests/*
    

    Then run the tests:

    coverage run --rcfile=coverage-options -m unittest discover tests
    

    The output would be:

    /tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
      self._warn("No data was collected.", slug="no-data-collected")
    .
    ----------------------------------------------------------------------
    Ran 1 test in 0.121s
    
    OK
    /tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
      self._warn("No data was collected.", slug="no-data-collected")
    

    The second warning is expected—actually, the test didn't get any coverage. The first warning, however, happens at the moment p (that is, multiprocessing.Process) terminates.

    One possible solution to get rid of this warning is to remove concurrency = multiprocessing from the coverage options file, or to change the value to a different library, such as concurrency = eventlet.