When running the following simple multiprocessing.Pool
code, the KeyboardInterrupt
does not stop the python program.
import multiprocessing as mp
import time
print(f"INIT {__name__}")
def worker(a):
time.sleep(999)
def main():
with mp.Pool(2) as p:
for _ in p.imap(worker, range(3)):
...
if __name__ == "__main__":
main()
INIT __main__
INIT __mp_main__
INIT __mp_main__
Process SpawnPoolWorker-2:
Process SpawnPoolWorker-1:
Traceback (most recent call last):
Traceback (most recent call last):
File "Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^^^^^
File "C:\WIN_PROJEKTE\informatik\synthetic-data-generation\src\test2.py", line 8, in worker
time.sleep(999)
KeyboardInterrupt
File "Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^^^^^
File "test2.py", line 8, in worker
time.sleep(999)
KeyboardInterrupt
INIT __mp_main__
INIT __mp_main__
As you can see, when I hit CTRL+C
, the two workers receive it and stop. But the main process appaerently does not receive it and the two multiprocessing threads are spawned again.
When I press CTRL+C
, the main process should stop.
Instead, the multiprocessing workers are stopped and spawned again.
When I hit CTRL+C
a second time, the same problem occurs
def worker(a):
return 1
def main():
with mp.Pool(2) as p:
for _ in p.imap(worker, range(100_000_000)):
pass
Works without issuesI found that catching the KeyboardInterrupt
in both the workers and main works correctly, but leave out either try/except and main continues to restart the workers.
import multiprocessing as mp
import time
print(f'INIT {__name__}')
def worker(a):
try:
time.sleep(999)
except KeyboardInterrupt:
print('worker ctrl-c')
def main():
try:
with mp.Pool(2) as p:
for _ in p.imap(worker, range(3)):
pass
except KeyboardInterrupt:
print('main ctrl-c')
if __name__ == '__main__':
main()
Output (hitting ctrl-c after the INITs):
INIT __main__
INIT __mp_main__
INIT __mp_main__
worker ctrl-c
worker ctrl-c
main ctrl-c