pythondebuggingpytestmonkeypatching

pdb.set_trace() fails inside test with monkeypatched open


I have a test monkeypatching builtin open to test a class which expects to read a file. When trying to debug it, I see:

TypeError: test_unique.<locals>.<lambda>() got an unexpected keyword argument 'encoding'
import builtins
import io
import pytest

def count_unique_letters_in_file(path: str):
    with open(path) as f:
        return len(set(c.lower() for c in f.read() if c.isalpha()))

def test_unique(monkeypatch: pytest.MonkeyPatch):
    text = 'Quick frog or something'
    monkeypatch.setattr(builtins, 'open', lambda _: io.StringIO(text))
    assert count_unique_letters_in_file('blah.txt') == 15

If I put import pdb: pdb.set_trace() anywhere after the monkeypatch line, or run pytest --pdb, then I get the above line. I understand it's because pdb is trying to use open which leads it to my lambda, but how do I work around it? Assume I cannot change the interface of tested function to accept an already open file or its contents.

Traceback is long as it includes all the test framework stuff up to the pdb.set_trace() or failed assert running with --pdb, but ends at:

self = <_pytest.debugging.pytestPDB._get_pdb_wrapper_class.<locals>.PytestPdbWrapper object at 0x123>
completekey = 'tab', stdin = None, stdout = None, skip = None, nosigint = False, readrc = True

def __init__(completekey='tab', stdin=None, stdout=None, skip=None, nosigint=False, readrc=True):
    ...
    if readrc:
        try:
            with open(os.path.expanduser('~/.pdbrc', encoding='utf-8') as rcFile:

Solution

  • You do not need to (and should not) mock or monkeypatch anything to test functionality that reads a file. Pytest in particular provides several fixtures for working with temporary directories, which let you easily create a real file for testing:

    import pathlib
    
    
    def count_unique_letters_in_file(path: str) -> int:
        with open(path) as f:
            return len(set(c.lower() for c in f.read() if c.isalpha()))
    
    
    def test_unique(tmp_path: pathlib.Path):
        path = tmp_path / 'blah.txt'
        path.write_text('Quick frog or something')
        assert count_unique_letters_in_file(str(path)) == 15
    

    For debugging purposes, the directories from the last 3 (by default) test runs are retained before pytest starts cleaning them up, and you can see the directory path (as with the value of any fixture) in the outputs if the test fails.


    In general, avoid mocking what you do not own - interfering with the builtins is, as you're seeing here, a particularly bad idea.