pythonunit-testingparameterized-unit-test

How do you generate dynamic (parameterized) unit tests in Python?


I have some kind of test data and want to create a unit test for each item. My first idea was to do it like this:

import unittest

l = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]

class TestSequence(unittest.TestCase):
    def testsample(self):
        for name, a,b in l:
            print "test", name
            self.assertEqual(a,b)

if __name__ == '__main__':
    unittest.main()

The downside of this is that it handles all data in one test. I would like to generate one test for each item on the fly. Any suggestions?


Solution

  • This is called "parametrization".

    There are several tools that support this approach. E.g.:

    The resulting code looks like this:

    from parameterized import parameterized
    
    class TestSequence(unittest.TestCase):
        @parameterized.expand([
            ["foo", "a", "a",],
            ["bar", "a", "b"],
            ["lee", "b", "b"],
        ])
        def test_sequence(self, name, a, b):
            self.assertEqual(a,b)
    

    Which will generate the tests:

    test_sequence_0_foo (__main__.TestSequence) ... ok
    test_sequence_1_bar (__main__.TestSequence) ... FAIL
    test_sequence_2_lee (__main__.TestSequence) ... ok
    
    ======================================================================
    FAIL: test_sequence_1_bar (__main__.TestSequence)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/site-packages/parameterized/parameterized.py", line 233, in <lambda>
        standalone_func = lambda *a: func(*(a + p.args), **p.kwargs)
      File "x.py", line 12, in test_sequence
        self.assertEqual(a,b)
    AssertionError: 'a' != 'b'
    

    For historical reasons I'll leave the original answer circa 2008):

    I use something like this:

    import unittest
    
    l = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]
    
    class TestSequense(unittest.TestCase):
        pass
    
    def test_generator(a, b):
        def test(self):
            self.assertEqual(a,b)
        return test
    
    if __name__ == '__main__':
        for t in l:
            test_name = 'test_%s' % t[0]
            test = test_generator(t[1], t[2])
            setattr(TestSequense, test_name, test)
        unittest.main()