I have a main function which I run with asyncio
and then inside it I used event_loop.run_in_executor()
to run mutliple processes of a blocking function.
What I wish to do is inside each of these processes to run a new asyncio loop for each one in order to execute async code.
So I have a main async function where I create multiple processes and I want to create a new asyncio loop in each of them. How can this be done?
You can just call asyncio.run
within your subprocess with a coroutine with the async work you want to do. A minimal example that spins up 5 processes each with their own event loops:
import asyncio
from concurrent.futures import ProcessPoolExecutor
def run_loop_in_process():
async def subprocess_async_work():
await asyncio.sleep(5) #whatever async code you need
asyncio.run(subprocess_async_work())
async def main():
loop = asyncio.get_running_loop()
pool = ProcessPoolExecutor()
tasks = [loop.run_in_executor(pool, run_loop_in_process) for _ in range(5)]
await asyncio.gather(*tasks)
if __name__ == "__main__":
asyncio.run(main())