So I want to use the queue multithreading in Python, where I queue jobs and it works through the jobs but I have two values I would like to pass to each job. Can I pass multiple values into the job queue, and how do you retrieve the data?
I tried the following:
def do_stuff(q, t):
while not q.empty():
value = q.get(0)
orgName = q.get(1)
slpTmr = random.randint(1, 3)
time.sleep(slpTmr)
print(str(value) + "[" + orgName + "] slept for: " + str(slpTmr))
q.task_done()
for org in orgs_search:
org_id = org["id"]
org_name = org["name"]
if org_id == 0:
continue
jobs.put(org_id, org_name)
for i in range(maxThreads):
worker = threading.Thread(target=do_stuff, args=(jobs))
worker.start()
but it doesn't print as expected, what I would hope for is an output of this:
123 [org1] slept for 2
142 [org2] slept for 4
123 [org3] slept for 3
342 [org4] slept for 1
You should be using the ThreadPoolExecutor()
or multiprocessing ThreadPool
library objects instead of rolling your own, but for the sake of exercise:
.get()
twice per item!)join()
the queue and the worker threads.import random
import threading
import time
from queue import Queue
def do_stuff(queue):
while not queue.empty():
value, orgName = queue.get()
slpTmr = random.uniform(1, 3)
time.sleep(slpTmr)
print(f"{value} [{orgName}] slept for {slpTmr}")
queue.task_done()
def main():
jobs = Queue()
orgs_search = [
{"id": 0, "name": "Apple"},
{"id": 1, "name": "Microsoft"},
{"id": 2, "name": "Google"},
{"id": 3, "name": "Disney"},
]
for org in orgs_search:
org_id = org["id"]
org_name = org["name"]
if org_id == 0:
continue
jobs.put((org_id, org_name))
max_threads = 2
workers = [threading.Thread(target=do_stuff, args=(jobs,)) for _ in range(max_threads)]
for worker in workers:
worker.start()
jobs.join()
print("Jobs are done!")
for worker in workers:
worker.join()
print("Workers are done!")
main()