pythonunit-testingtestingpytestpytest-markers

How can I run only tests with specific marker attribute in Pytest?


I'm using @pytest.mark to uniquely identify specific tests, therefore I created my custom marker.

@pytest.mark.key

I'm using it as such:

@pytest.mark.key("test-001")
def test_simple(self):
    self.passing_step()
    self.passing_step()
    self.passing_step()
    self.passing_step()
    assert True

Now from the console I would like to run all tests with the marked key "test-001". How can I achieve this?

What i'm looking for is something like this:

pypi.org/project/pytest-jira/0.3.6

where a test can be mapped to a Jira key. I looked at the source code for the link but i'm unsure how to achieve it in order for me to run specific tests. Say I only wanna run the test with the key "test-001".


Solution

  • Pytest does not provide this out of the box. You can filter by marker names using the -m option, but not by marker attributes.
    You can add your own option to filter by keys, however. Here is an example:

    conftest.py

    def pytest_configure(config):
        # register your new marker to avoid warnings
        config.addinivalue_line(
            "markers",
            "key: specify a test key"
        )
    
    
    def pytest_addoption(parser):
        # add your new filter option (you can name it whatever you want)
        parser.addoption('--key', action='store')
    
    
    def pytest_collection_modifyitems(config, items):
        # check if you got an option like --key=test-001
        filter = config.getoption("--key")
        if filter:
            new_items = []
            for item in items:
                mark = item.get_closest_marker("key")
                if mark and mark.args and mark.args[0] == filter:
                    # collect all items that have a key marker with that value
                    new_items.append(item)
            items[:] = new_items
    

    Now you run something like

    pytest --key=test-001
    

    to only run the tests with that marker attribute.

    Note that this will still show the overall number of tests as collected, but run only the filtered ones. Here is an example:

    test_key.py

    import pytest
    
    @pytest.mark.key("test-001")
    def test_simple1():
        pass
    
    @pytest.mark.key("test-002")
    def test_simple2():
        pass
    
    
    @pytest.mark.key("test-001")
    def test_simple3():
        pass
    
    def test_simple4():
        pass
    
    
    $ python -m pytest -v --key=test-001 test_key.py
    
    ...
    collected 4 items
    
    test_key.py::test_simple1 PASSED
    test_key.py::test_simple3 PASSED
    ================================================== 2 passed in 0.26s ==================================================
    

    UPDATE for pytest >= 8.3

    As mentioned by MAndrĂ¡s in the comments, support for keyword arguments in markers has been added in pytest 8.3.
    It will not work for positional arguments as used in the example above, but if rewriting the markers to use keyword arguments, e.g.:

    import pytest
    
    @pytest.mark.key(name="test-001")
    def test_simple1():
        pass
    
    @pytest.mark.key(name="test-002")
    def test_simple2():
        pass
    
    ...
    

    pytest now supports filtering by these arguments out of the box:

    $ python -m pytest -v -m key(name=\"test-001\") test_key.py
    ...
    collected 4 items
    
    test_key.py::test_simple1 PASSED
    test_key.py::test_simple3 PASSED
    ================================================== 2 passed in 0.28s ==================================================
    

    Note that the apostrophes have to be escaped for the command line to be correctly passed.