I am trying to write unit tests that involve mocking several libraries in different ways for each test. When I run each test individually, they all pass, but when I run them all together, many of them fail. The reason they fail is that when a method is mocked in two tests, the mock from the second test is ignored, and the mocked function always behaves as defined by the first test.
I have made a demo project to demonstrate the issue:
/.
├── src
│ └── main.py
└── test
├── __init__.py
├── test_base.py
└── test.py
main.py
import boto3
s3 = boto3.client('s3')
def main():
return s3.list_buckets()
__init__.py
from .test import *
test_base.py
import unittest
from unittest.mock import MagicMock, patch
class TestBase(unittest.TestCase):
def setUp(self):
self.patch_boto3_client = patch('boto3.client', autospec=True)
self.mock_boto3_client = self.patch_boto3_client.start()
# BOTO3
# Set up mock boto3 clients
self.mock_s3 = MagicMock()
# Configure mock_boto3_client to return our mock clients
self.mock_boto3_client.side_effect = lambda service: {
's3': self.mock_s3,
}[service]
def tearDown(self) -> None:
self.patch_boto3_client.stop()
test.py
from .test_base import TestBase
class Test(TestBase):
def test_1(self):
from src.main import main
self.mock_s3.list_buckets.return_value = 'test_1'
response = main()
self.assertEqual(response, 'test_1')
def test_2(self):
from src.main import main
self.mock_s3.list_buckets.return_value = 'test_2'
response = main()
self.assertEqual(response, 'test_2')
$ python3 -m coverage run -m unittest test
.F
======================================================================
FAIL: test_2 (test.test.Test.test_2)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../test/test.py", line 21, in test_2
self.assertEqual(response, 'test_2')
AssertionError: 'test_1' != 'test_2'
- test_1
? ^
+ test_2
? ^
----------------------------------------------------------------------
Ran 2 tests in 0.292s
FAILED (failures=1)
I have tried patching by using decorators, and manually starting and stopping patch objects as shown in my code.
How can I make my unit tests respect updates I make to mocked functions?
The problem in your file test.py
is that in the second test method (test_2()
) the import instruction:
from src.main import main
doesn't cause the execution of the instruction:
s3 = boto3.client('s3')
present in main.py
, so s3
in main.py
remains equal to the Mock object created in the first test.
Instead by test_1()
is executed the instruction s3 = boto3.client('s3')
as effect of the import instruction from src.main import main
and to s3
is assigned the Mock object mock_s3
created by the method setUp()
.
The second test could be modified as following (comments highlight the changes):
from .test_base import TestBase
from unittest import mock # <--- ADD this import
class Test(TestBase):
def test_1(self):
...
def test_2(self):
from src.main import main, s3 # <--- ADD HERE IMPORT of s3 for patch()
# remove your return_value
#self.mock_s3.list_buckets.return_value = 'test_2'
# ADD the following with mock.patch(...) instruction
with mock.patch('src.main.s3') as mock_s3:
mock_s3.list_buckets.return_value = 'test_2'
response = main()
self.assertEqual(response, 'test_2')
The execution of the test gives the following output:
$ python3 -m coverage run -m unittest test
..
----------------------------------------------------------------------
Ran 2 tests in 0.131s
OK