pythonpython-3.xwindowspickleread-write

Python "r+" mode reads nothing after writing with Pickle library


I have a file that I want to read a list from and then write an updated list to it by using the pickle library. But after that it reads nothing from the file. I have tried using other people's examples, even copy-pasted some, but I cannot get it to work.

I am using Windows 10 and am aware of the bug that was mentioned in similar questions. The solution I got from those questions was to add f.seek(f.tell()) before writing, but that does not work when I'm writing with the pickle library.

Here's what I have:

import pickle

with open("data.bin", "r+b") as file:
    temp_data = pickle.load(file)
    temp_data.append("test")
    file.seek(file.tell())
    pickle.dump(temp_data, file)

with open("data.bin", "rb") as file:
    print(pickle.load(file))
    # prints empty list

If I read and write separately, then it does work. But I should be able to use r+ right? (I have also tried all this with a regular text file in case it had to do with binary mode, but I got the same results.)

import pickle

temp_data = []

with open("data.bin", "rb") as file:
    temp_data = pickle.load(file)

with open("data.bin", "wb") as file:
    temp_data.append("test")
    pickle.dump(temp_data, file)

with open("data.bin", "rb") as file:
    print(pickle.load(file))
    # prints ['test']

Does anyone know why this happens, or do I just have to do it separately?


Solution

  • Remember that both reading and writing by definition moves the file handle's position to the end of the file, so you need to set that back to zero every time you intend to "do more pickle related things later":

    import os, pickle
    
    fname = "data.bin"
    data = ["test"]
    
    if not os.path.isfile(fname):
        with open(fname, "wb") as file:
            pickle.dump(data, file)
            file.close()
        print(f"{fname} created")
    
    with open(fname, "r+b") as file:
        # load our file and reset the file pointer for future pickles
        data = pickle.load(file)
        file.seek(0)
    
        print("data read:", data)
    
        # update our data...
        data.append(f"test-{len(data)}")
        print("updated data:", data)
    
        # ...then write that back, again resetting the file pointer afterwards
        pickle.dump(data, file)
        file.seek(0)
    
        # then let's load that back in, still remembering that file pointer
        data = pickle.load(file)
        file.seek(0)
    
        print("reloaded data:", data)
    
        # ...and then the rest of what you need to do...
    

    Running this three times:

    (venv) D:\temp>python test.py       
    data.bin created
    data read: ['test']
    updated data: ['test', 'test-1']
    reloaded data: ['test', 'test-1']
    
    (venv) D:\temp>python test.py
    data read: ['test', 'test-1']
    updated data: ['test', 'test-1', 'test-2']
    reloaded data: ['test', 'test-1', 'test-2']
    
    (venv) D:\temp>python test.py
    data read: ['test', 'test-1', 'test-2']
    updated data: ['test', 'test-1', 'test-2', 'test-3']
    reloaded data: ['test', 'test-1', 'test-2', 'test-3']