pythonpickleoserror

How to prevent data loss when writing to a file fails with "no space left on device"?

I use pickle to save a Python dictionary to a file.

with open(FILENAME, 'wb') as f:
    pickle.dump(DATA, f, protocol=pickle.HIGHEST_PROTOCOL)

It was all good until the disk run out of space on a shared server and my file became empty (0 byte).

Traceback (most recent call last):
  File "****.py", line 81, in ****
    with open(FILENAME, 'wb') as f:
OSError: [Errno 28] No space left on device

What is the best solution to prevent overwriting the previous data if the above error occurs?

Solution

  • Write to a temporary file (on the same filesystem!), and move it to the real destination when finished. Maybe with a fsync in between, to make sure the new data is really written.