I have a test suite that runs a bunch of system tests on an API. Some system tests interact with the API by using Flask's test_client
, whereas a few specific tests interact with the API by making HTTP queries to a Gunicorn running the API.
When I use coverage
, as soon as it hits those special tests that use Gunicorn, a warning is shown:
CoverageWarning: No data was collected. (no-data-collected)
The warning makes perfect sense—coverage
cannot track the actual coverage in tests when the tested API runs through Gunicorn, instead of being called directly. However, it's annoying to see this warning at every test run.
How do I hide it for those specific tests, while ensuring it will be shown if the coverage for other tests starts misbehaving?
A few additional details:
All tests (both the ones that rely on Flask's test_client
and the ones that do HTTP queries to Gunicorn) are run in one go with the command:
coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
The contents of the aforementioned coverage-options
file are:
[run]
branch = True
concurrency = multiprocessing
omit =
/usr/*/dist-packages/*
*/.local/lib/*/site-packages/*
tests/*
The tests/
directory contains several sub-directories:
tests/api/
: about one hundred and thirty tests that rely on test_client
.tests/system/
: four tests that perform HTTP queries to Gunicorn.tests/unit/
: seven very basic tests.The output is:
(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...................................................................................................................................../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
self._warn("No data was collected.", slug="no-data-collected")
.../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
self._warn("No data was collected.", slug="no-data-collected")
.......
----------------------------------------------------------------------
Ran 143 tests in 45.853s
OK
As tests are ran in alphabetical order, the first one hundred and thirty something dots match the tests in tests/api/
. The last seven correspond to the unit tests. The warnings are shown once the very first test from tests/system/
gets executed, and then once again when all four are executed.
By comparison, if I remove tests/system/
directory, this is what I get:
(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...........................................................................................................................................
----------------------------------------------------------------------
Ran 139 tests in 38.022s
OK
Trying to create a minimal reproducible example, it appears that the warning is emitted every time there is a multiprocessing.Process
in the test, and more precisely when this process terminates.
Here are the steps to get the warning.
mkdir /tmp/ndc
cd /tmp/ndc
python3 -m venv .venv
source .venv/bin/activate
pip install coverage
Create the following tree in /tmp/ndc
:
tests/
__init__.py (empty)
test_demo.py ①
coverage-options ②
Contents of /tmp/ndc/tests/test_demo.py
①:
import multiprocessing
import time
import unittest
class DemoTests(unittest.TestCase):
def test_with_multiprocessing(self):
p = multiprocessing.Process(target=t)
p.start()
p.join()
self.assertTrue(2 * 2 == 4)
def t():
time.sleep(0.1)
Contents of /tmp/ndc/coverage-options
②:
[run]
branch = True
concurrency = multiprocessing
omit =
/usr/*/dist-packages/*
*/.local/lib/*/site-packages/*
tests/*
Then run the tests:
coverage run --rcfile=coverage-options -m unittest discover tests
The output would be:
/tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
self._warn("No data was collected.", slug="no-data-collected")
.
----------------------------------------------------------------------
Ran 1 test in 0.121s
OK
/tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
self._warn("No data was collected.", slug="no-data-collected")
The second warning is expected—actually, the test didn't get any coverage. The first warning, however, happens at the moment p
(that is, multiprocessing.Process
) terminates.
One possible solution to get rid of this warning is to remove concurrency = multiprocessing
from the coverage options file, or to change the value to a different library, such as concurrency = eventlet
.