pythonunit-testingmockingpython-unittestpython-unittest.mock

When writing unit tests in python, why are mocks in subsequent tests not overwriting mocks made in previous tests?


I am trying to write unit tests that involve mocking several libraries in different ways for each test. When I run each test individually, they all pass, but when I run them all together, many of them fail. The reason they fail is that when a method is mocked in two tests, the mock from the second test is ignored, and the mocked function always behaves as defined by the first test.

I have made a demo project to demonstrate the issue:

File Structure

/.
├── src
│   └── main.py
└── test
    ├── __init__.py
    ├── test_base.py
    └── test.py

Files

main.py

import boto3

s3 = boto3.client('s3')

def main():
    return s3.list_buckets()

__init__.py

from .test import *

test_base.py

import unittest
from unittest.mock import MagicMock, patch

class TestBase(unittest.TestCase):
    def setUp(self):
        self.patch_boto3_client = patch('boto3.client', autospec=True)

        self.mock_boto3_client = self.patch_boto3_client.start()

        # BOTO3

        # Set up mock boto3 clients
        self.mock_s3 = MagicMock()
        
        # Configure mock_boto3_client to return our mock clients
        self.mock_boto3_client.side_effect = lambda service: {
            's3': self.mock_s3,
        }[service]

    def tearDown(self) -> None:
        self.patch_boto3_client.stop()

test.py

from .test_base import TestBase

class Test(TestBase):
    def test_1(self):
        from src.main import main
        
        self.mock_s3.list_buckets.return_value = 'test_1'

        response = main()

        self.assertEqual(response, 'test_1')

    
    def test_2(self):
        from src.main import main
        
        self.mock_s3.list_buckets.return_value = 'test_2'

        response = main()

        self.assertEqual(response, 'test_2')

Results

$ python3 -m coverage run -m unittest test
.F
======================================================================
FAIL: test_2 (test.test.Test.test_2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../test/test.py", line 21, in test_2
    self.assertEqual(response, 'test_2')
AssertionError: 'test_1' != 'test_2'
- test_1
?      ^
+ test_2
?      ^


----------------------------------------------------------------------
Ran 2 tests in 0.292s

FAILED (failures=1)

I have tried patching by using decorators, and manually starting and stopping patch objects as shown in my code.

How can I make my unit tests respect updates I make to mocked functions?


Solution

  • The problem in your file test.py is that in the second test method (test_2()) the import instruction:

    from src.main import main
    

    doesn't cause the execution of the instruction:

    s3 = boto3.client('s3')
    

    present in main.py, so s3 in main.py remains equal to the Mock object created in the first test.

    Instead by test_1() is executed the instruction s3 = boto3.client('s3') as effect of the import instruction from src.main import main and to s3 is assigned the Mock object mock_s3 created by the method setUp().

    Workaround

    The second test could be modified as following (comments highlight the changes):

    from .test_base import TestBase
    from unittest import mock    # <--- ADD this import
    
    class Test(TestBase):
       def test_1(self):
            ...
    
       def test_2(self):
            from src.main import main, s3  # <--- ADD HERE IMPORT of s3 for patch()
    
            # remove your return_value
            #self.mock_s3.list_buckets.return_value = 'test_2'
    
            # ADD the following with mock.patch(...) instruction
            with mock.patch('src.main.s3') as mock_s3:
                mock_s3.list_buckets.return_value = 'test_2'
                response = main()
                self.assertEqual(response, 'test_2')
    

    The execution of the test gives the following output:

    $ python3 -m coverage run -m unittest test
    ..
    ----------------------------------------------------------------------
    Ran 2 tests in 0.131s
    
    OK