Setting os.environ["TMP"] works in the code but not in pytest tests

I have the following python code:

import tempfile
import os

def test_paul_confused():
    os.environ["TMP"] = "/home/fedora"
    assert tempfile.gettempdir() == "/home/fedora"
    print(tempfile.gettempdir())

test_paul_confused()

When I run this code from the command line I get the expected result:

(fedora) [fedora@ip-10-254-6-135 ~]$ uv run pytest.py
/home/fedora

However when I run the code via pytest, the TMP directory doesn't get set correctly!

(fedora) [fedora@ip-10-254-6-135 ~]$ uv run pytest  pytest.py
============================ test session starts =============================
platform linux -- Python 3.13.5, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/fedora
collected 0 items / 1 error

=================================== ERRORS ===================================
_________________________ ERROR collecting pytest.py _________________________
pytest.py:9: in <module>
    test_paul_confused()
pytest.py:6: in test_paul_confused
    assert tempfile.gettempdir() == "/home/fedora"
E   AssertionError: assert '/tmp' == '/home/fedora'
E    +  where '/tmp' = <function gettempdir at 0xffffb3d83d80>()
E    +    where <function gettempdir at 0xffffb3d83d80> = tempfile.gettempdir
========================== short test summary info ===========================
ERROR pytest.py - AssertionError: assert '/tmp' == '/home/fedora'
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.07s ==============================
(fedora) [fedora@ip-10-254-6-135 ~]$

Why is this code behaving differently in testing than when it's run outside of testing?

Solution

After the first call to tempfile.gettempdir() the value it reads is cached - meaning that if you change the value in os.environ dictionary after the first call in the process, the value returned first is still returned:

Python 3.13.2 (main, Feb  4 2025, 00:00:00) [GCC 14.2.1 20250110 (Red Hat 14.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> tempfile.gettempdir()
'/tmp'
>>> os.environ["TMP"] = "/tmp/xis"
>>> tempfile.gettempdir()
'/tmp'

(While if I set TMP first, that works).

So, if there is any call to tempfile.gettempdir(), and in fact, to any tempdir method in the test process before running the code in your function, the test as it is will fail.

Also, see in the docs that both the TMPDIR and TEMP environment variables work the same, but have precedence over TMP: meaning that if the testing process has any of those set, changing TMP won't work.

There are a couple calls marked as private on the tempdir module, but I found none which could control this cached value and 'reset' it, so that the tempfile setting is deterministic - (the tempfile._candiate_tempdir_list() call is dynamic and will always return a list with the current candidate directories for tempfiles, in order of priority, and you can change the TMP, TEMP, TMPDIR variables and repeat calls to this to see the list changing - however, both tempfile.gettempdir() and the inner tempfile._gettempdir() won't change after the first call to either, or any method which actually creates a tempfile or tempdir (as they probably use these internally)

Sorry, but I can't find a deterministic way out for you short of setting the TMP var at the OS level, before calling pytest