pythonunit-testingtestingpytestxdist

pytest: How to get a list of all failed tests at the end of the session? (and while using xdist)


I would like to have a list of all the tests that have failed to be used at the end of session.

Pytest lets you define a hook pytest_sessionfinish(session, exitstatus), that is called at the end of the session, where I wish to have that list.

session is a _pytest.main.Session instance that has the attribute items (type list), but I couldn't find whether the each item in that list passed of failed.

  1. How can a list of all failed tests could be retrieved at the end of the session?
  2. How can it be done while using pytest-xdist plugin, where I would like to get that list in the master process. Using this plugin, session does not even have items attribute in the master:

    def pytest_sessionfinish(session, exitstatus):
        if os.environ.get("PYTEST_XDIST_WORKER", "master") == "master":
             print(hasattr(session, "items"))  # False
    

Solution

  • If you want results of the tests you can use hook runtest_makereport:

    @pytest.hookimpl(tryfirst=True, hookwrapper=True)
    def pytest_runtest_makereport(item, call):
        outcome = yield
        rep = outcome.get_result()
        if rep.when == 'call' and rep.failed:
            mode = 'a' if os.path.exists('failures') else 'w'
            try:  # Just to not crash py.test reporting
              pass  # the test 'item' failed
            except Exception as e:
                pass