pythonfileunixposixatomic

Create a new file, moving an existing file out of the way if needed

What is the best way to create a new file in Python, moving if needed an existing file with the same name to a different path?

While you could do

if os.path.exists(name_of_file):
   os.move(name_of_file, backup_name)
f = open(name_of_file, "w")

that has TOCTOU issues (e.g. multiple processes could try and create the file at the same time). Can I avoid those issues using only the standard library (preferably), or is there a package which handles this.

You can assume a POSIX file system.

Solution

  • In a loop:

    Open the file exclusively (open(..., "x") or O_CREAT|O_EXCL).

    If that fails with FileExistsError (EEXIST), then atomically os.rename the existing file to something else. Try again.

    If that renaming fails with anything other than FileExistsError (ENOENT, meaning someone else removed or renamed the offending file before you did), break the loop and fail.

    (It's not clear to me how your competing processes can know it is OK to move the existing file out of the way or not, but presumably you've knowledge of your use case that I do not.)