python-3.xpython-asyncio

How can I integrate asyncio with an external event loop?


I'm writing an event-driven program with the event scheduling written in C. The program uses Python for extension modules. I would like to allow extension modules to use async/await syntax to implement coroutines. Coroutines will just interact with parts of my program, no IO is involved. My C scheduler is single-threaded and I need coroutines to execute in its thread.

In a pure Python program, I would just use asyncio as is, and let my program use its event loop to drive all events. This is however not an option; my event loop needs to serve millions of C-based events per second and I cannot afford Python's overheads.

I tried to write my own event loop implementation, that delegates all scheduling to my C scheduler. I tried a few approaches:

Any ideas on how to best do this? How did others solve this problem?


Solution

  • I found that trio, a third-party alternative to asyncio, provides explicit support for integration with alien event loops through something called guest mode. Solves my problem!