pythonmultithreadingsemaphorecross-process

Named semaphores in Python?


I have a script in python which uses a resource which can not be used by more than a certain amount of concurrent scripts running.

Classically, this would be solved by a named semaphores but I can not find those in the documentation of the multiprocessing module or threading .

Am I missing something or are named semaphores not implemented / exposed by Python? and more importantly, if the answer is no, what is the best way to emulate one?

Thanks, Boaz

PS. For reasons which are not so relevant to this question, I can not aggregate the task to a continuously running process/daemon or work with spawned processes - both of which, it seems, would have worked with the python API.


Solution

  • I suggest a third party extension like these, ideally the posix_ipc one -- see in particular the semaphore section in the docs.

    These modules are mostly about exposing the "system V IPC" (including semaphores) in a unixy way, but at least one of them (posix_ipc specifically) is claimed to work with Cygwin on Windows (I haven't verified that claim). There are some documented limitations on FreeBSD 7.2 and Mac OSX 10.5, so take care if those platforms are important to you.