suppose i have a dictionary such as
dict = {A:[1,2,3], B:[4,5,6], C:[7,8,9], ......}
I want to process each element of a particular key's list one after other but the individual keys could be processed in parallel using concurrent.futures.ThreadPoolExecutor
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as ex:
ex.submit(process_element, contents of A)
ex.submit(process_element, contents of B)
ex.submit(process_element, contents of C)
.
.
.
So the output should be
result of process_element A[0]
result of process_element B[0]
result of process_element C[0]
.
.
.
result of process_element A[1]
but not necessary in that order
what is wrong with the above method ?
It doesn't look like you're passing the list, but instead a variable named 'A'. I'm not sure when you run your code why your not getting an error. What you want is below
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as ex:
ex.submit(process_element, dict['A'])
ex.submit(process_element, dict['B'])
ex.submit(process_element, dict['C'])
To process each element one by one, then you could simply use a list construction:
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as ex:
procsA = [ex.submit(process_element, elem) for elem in dict['A']
...
...