pythonasynchronousasync-awaitpython-asyncioaiohttp

How to await method in loop?


I have the following method that generates data for me:

async def generate_url(self, video_id):
    data = await self.s3.generate_presigned_url(...video_id...)
    return data

def convert_to_json(self, urls):
    ids = [self.generate_url(url) for url in urls]
    ...

How to correctly await generate_url in convert_to_json?


Solution

  • You may use a wait wrapper around the list of tasks:

    async def convert_to_json(self, urls):
        tasks = [self.generate_url(url) for url in urls]
        await asyncio.wait(tasks)
    

    Or, if you can't mark convert_to_json method as async, wait it synchronously:

    import asyncio
    
    def convert_to_json(self, urls):
        loop = asyncio.get_event_loop()
        tasks = [self.generate_url(url) for url in urls]
        loop.run_until_complete(asyncio.wait(tasks))
    

    Also you can try to implement an async iterator and use it with async for syntax, something like this:

    class Loader:
        def __init__(self, urls):
            self._urls = iter(urls)
    
        async def generate_url(self, video_id):
            data = await self.s3.generate_presigned_url(...video_id...)
            return data
    
        def __aiter__(self):
            return self
    
        async def __anext__(self):
            try:
                url = next(self._urls)
            except StopIteration:
                raise StopAsyncIteration
            data = await self.generate_url(url)
            return data
    
    async for id in Loader(urls):
        print(id)