[I am having an extremely difficult time to implement a thread-/process-safe solution to acquire a file lock using python 3 on Linux (I do not care about portable solutions as the program I am working on makes extensive use of Linux-kernel-exclusive-containerization technologies).]
After reading http://apenwarr.ca/log/?m=201012#13 , I decided to use fcntl.lockf()
to lock a file for process-exclusive access and wrote the following function:
import contextlib as Contextlib
import errno as Errno
import fcntl as Fcntl
import os as Os
@Contextlib.contextmanager
def exclusiveOpen(filename,
mode):
try:
fileDescriptor = Os.open(filename,
Os.O_WRONLY | Os.O_CREAT)
except OSError as e:
if not e.errno == Errno.EEXIST:
raise
try:
Fcntl.lockf(fileDescriptor,
Fcntl.LOCK_EX)
fileObject = Os.fdopen(fileDescriptor,
mode)
try:
yield fileObject
finally:
fileObject.flush()
Os.fdatasync(fileDescriptor)
finally:
Os.close(fileDescriptor)
Apart from that I am certain, that it is incorrect (why doesn't it block in Fcntl.lockf(fileDescriptor, Fcntl.LOCK_EX)
?), the part which makes me feel uneasy the most, is where the fileDescriptor
is acquired - if the file is non existent, it is created ... but what is going on, if two processes execute this part simultaneously? Isn't there a chance of a race condition, where both threads attempt to create the file? And if so, how could one possibly prevent that - certainly not with another lock file (?) because it would have to be created in the same manner (?!?!) I'm lost. Any help is greatly appreciated.
UPDATE: Posted another approach to the underlying problem. The problem I see in this approach, is that a procedure name must not equal the name of an existent UNIX domain socket (possibly created by another program) - am I correct with this?
AFAIK, there is no possible race condition when creating a file. If two (or more) processes or threads try to simultaneously create the same file by open
with the O_CREAT
flag and not the O_EXCL
one, the file will be created once, and all callers will get an open file descriptor on the same file - exactely what you need. I assume that you are using C-Python on Linux, and that your Python code will end using the underlying C open function.
So when your code executes the lockf
function, you have a opened file descriptor on an existing file, so the underlying lockf
call guarantees that only one process can hold the lock at a time, provided the underlying file system supports locking.
I tried it on a Unix (FreeBSD) system and it works as expected.