pythonasynchronouspython-asyncio

Process list of async tasks immediately upon each task's completion

How can I process the results of a list of async tasks immediately as a task is completed?

For instance, the following should display whichever page loads first:

urls = ['stackoverflow.com', 'google.com']
tasks = [asyncio.create_task(fetch_page(x)) for x in urls]

for page in asyncio.give_me_results_ASAP(tasks):
    print(page.url)

Since google loads faster, I'd like it to print:

google.com
stackoverflow.com
Solution

  • asyncio.as_completed is designed for exactly this, and returns an iterator of coroutines in the order that the tasks are completed. The first coroutine returned from the iterator will correspond to the first task that completes, and you can await on each coroutine to get the result of the task.

    # With Python 3.8+
import asyncio
import time

async def fetch_page(url):
    reponse_time = 0.1 if url == 'google.com' else 0.8
    await asyncio.sleep(reponse_time)
    return url

async def main():
    urls = ['stackoverflow.com', 'google.com']
    tasks = [asyncio.create_task(fetch_page(x)) for x in urls]

    for coro in asyncio.as_completed(tasks):
        print(f"{time.time():.3f}", await coro)

if __name__ == '__main__':
    asyncio.run(main())

    produces:

    1600821288.178 google.com
1600821288.280 stackoverflow.com