Why does writing to %appdata% from the Windows Store version of Python not work?

I was trying to write some data to %appdata%. Everything seemed to work as shown in the output of Script1. The new directories are being created and the file is saved and the data gets retrieved successfully as well. But when trying to look at the data in File Explorer, the folder wasn't there! CMD couldn't find the file and directory either.

Later I created the file manually and checked what happened. The CMD could now find the file (which I just manually created), but when trying to read the file with Python, it'd output me the Python ghost file contents test data 123 and not what I'd just written into it! I also double-checked with WSL that the new file actually contains test data 456.

Why is this happening, and how can I resolve it?

Script1 (Creating the file with test data 123):

import os
import subprocess


appdata        = os.getenv('APPDATA')
directory_path = f"{appdata}\\com-company\\prod-product-version3"
file_path      = directory_path + "\\file1.txt"


print(f"Directories Exist: {os.path.exists(directory_path)}")
if not os.path.exists(directory_path):
    os.makedirs(directory_path)
    print("Directories created")
print(f"Directories Exist: {os.path.exists(directory_path)}")



print(f"File Exist: {os.path.exists(file_path)}")
print(f"Writing File: {file_path}")
with open(file_path, 'w')as fp:
    fp.write("test data 123")
print(f"File Exist: {os.path.exists(file_path)}")


print(f"Reading File: {file_path}")
with open(file_path, 'r')as fp:
    print(f"File Content: {fp.read()}")



print('---------------------')
cmd = f"dir {directory_path}"
try:
    output = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT, text=True)
    print(output)
except subprocess.CalledProcessError as e:
    print(f'Error: {e}')
    print(f'Error message:\n{e.output}')


print('---------------------')
cmd = f"dir {file_path}"
try:
    output = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT, text=True)
    print(output)
except subprocess.CalledProcessError as e:
    print(f'Error: {e}')
    print(f'Error message:\n{e.output}')

Output:

Directories Exist: False
Directories created
Directories Exist: True
File Exist: False
Writing File: C:\Users\one\AppData\Roaming\com-company\prod-product-version3\file1.txt
File Exist: True
Reading File: C:\Users\one\AppData\Roaming\com-company\prod-product-version3\file1.txt
File Content: test data 123
---------------------
Error: Command 'dir C:\Users\one\AppData\Roaming\com-company\prod-product-version3' returned non-zero exit status 1.
Error message:
The system cannot find the file specified.

---------------------
Error: Command 'dir C:\Users\one\AppData\Roaming\com-company\prod-product-version3\file1.txt' returned non-zero exit status 1.
Error message:
The system cannot find the path specified.

Creating C:\Users\one\AppData\Roaming\com-company\prod-product-version3\file1.txt manually and writing data into it:

test data 456

Script2 (reading test data 123 even though it contains test data 456):

import os
appdata        = os.getenv('APPDATA')
directory_path = f"{appdata}\\com-company\\prod-product-version3"
file_path      = directory_path + "\\file1.txt"

print(f"File Exist: {os.path.exists(file_path)}")

print(f"Reading File: {file_path}")
with open(file_path, 'r')as fp:
    print(f"File Content: {fp.read()}")

Output:

File Exist: True
Reading File: C:\Users\one\AppData\Roaming\com-company\prod-product-version3\file1.txt
File Content: test data 123

Double checking with WSL:

cat /mnt/c/Users/one/AppData/Roaming/com-company/prod-product-version3/file1.txt
Output: test data 456

PS: I rebooted my system and python still thinks the file contains test data 123. And writing normally works just fine:

with open('C:\\Users\\one\\Desktop\\file2.txt', 'w') as fp:
    fp.write('test data 789')

This is a bug in the Windows Store version of Python that won't be fixed, as the developers have deprecated the Store version and are pushing users to use the Python modules installer, which won't have this limitation¹.

It's still present, even with Python 3.12 installed from the Microsoft Store. Using procmon and some blackbox testing I found the following:

Any paths under C:\Users\username\AppData\REST-OF-THE-PATH are actually treated as C:\Users\username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\REST-OF-THE-PATH.

This happens regardless of working in a venv or not, and regardless of the library call you're using (i.e. pathlib and open behave the same)

---

^{1: See https://github.com/python/cpython/issues/122057#issuecomment-2930145493}