pythonmultithreadingpython-asyncioeventqueue

python avoid busy wait in event processing thread


How can I avoid busy_wait from the event consumer thread using asyncio? I have a main thread which generates events which are processed by other thread. My event thread has busy_wait as it is trying to see if event queue has some item in it...

from Queue import Queue

from threading import Thread
import threading

def do_work(p):
    print("print p - %s %s" % (p, threading.current_thread()))

def worker():
    print("starting %s" % threading.current_thread())
    while True: # <------------ busy wait
        item = q.get()
        do_work(item)
        time.sleep(1)
        q.task_done()

q = Queue()
t = Thread(target=worker)
t.daemon = True
t.start()

for item in range(20):
    q.put(item)

q.join()       # block until all tasks are done

How can I achieve something similar to the above code using asyncio?


Solution

  • asyncio makes sense only if you are working with IO, for example running an HTTP server or client. In the following example asyncio.sleep() simulates I/O calls. If you have a bunch of I/O tasks it can get as simple as:

    import asyncio
    
    import random
    
    async def do_work(i):
        print("[#{}] work part 1".format(i))
        await asyncio.sleep(random.uniform(0.5, 2))
        print("[#{}] work part 2".format(i))
        await asyncio.sleep(random.uniform(0.1, 1))
        print("[#{}] work part 3".format(i))
        return "#{}".format(i)
    
    
    loop = asyncio.get_event_loop()
    tasks = [do_work(item + 1) for item in range(20)]
    print("Start...")
    results = loop.run_until_complete(asyncio.gather(*tasks))
    print("...Done!")
    print(results)
    loop.close()
    

    see also ensure_future and asyncio.Queue.