I'm trying to use ThreadPoolExecutor
in Python 3.6 on Windows 7 and it seems that the exceptions are silently ignored or stop program execution. Example code:
#!/usr/bin/env python3
from time import sleep
from concurrent.futures import ThreadPoolExecutor
EXECUTOR = ThreadPoolExecutor(2)
def run_jobs():
EXECUTOR.submit(some_long_task1)
EXECUTOR.submit(some_long_task2, 'hello', 123)
return 'Two jobs was launched in background!'
def some_long_task1():
print("Task #1 started!")
for i in range(10000000):
j = i + 1
1/0
print("Task #1 is done!")
def some_long_task2(arg1, arg2):
print("Task #2 started with args: %s %s!" % (arg1, arg2))
for i in range(10000000):
j = i + 1
print("Task #2 is done!")
if __name__ == '__main__':
run_jobs()
while True:
sleep(1)
The output:
Task #1 started!
Task #2 started with args: hello 123!
Task #2 is done!
It's hanging there until I kill it with Ctrl+C.
However, when I remove 1/0
from some_long_task1
, Task #1 completes without problem:
Task #1 started!
Task #2 started with args: hello 123!
Task #1 is done!
Task #2 is done!
I need to capture the exceptions raised in functions running in ThreadPoolExecutor
somehow.
Python 3.6 (Minconda), Windows 7 x64.
ThreadPoolExecutor.submit
returns a future object that represents the result of the computation, once it becomes available. In order to not ignore exceptions raised by the submitted function, you need to actually access this result. First, you can change run_job
to return the created futures:
def run_jobs():
fut1 = EXECUTOR.submit(some_long_task1)
fut2 = EXECUTOR.submit(some_long_task2, 'hello', 123)
return fut1, fut2
Then, have the top-level code wait for the futures to complete, and access their results:
import concurrent.futures
if __name__ == '__main__':
futures = run_jobs()
concurrent.futures.wait(futures)
for fut in futures:
print(fut.result())
Calling result()
on a future whose execution raised an exception will propagate the exception to the caller. In this case the ZeroDivisionError
will get raised at top-level.
Of course, instead of allowing the exception to be raised and propagated, you can place the result()
invocation in a try
block and catch
to catch the exception. Alternatively, you can call the exception()
method on the Future
object to just retrieve the raised exception, if any.