I'm writing an event-driven program with the event scheduling written in C. The program uses Python for extension modules. I would like to allow extension modules to use async
/await
syntax to implement coroutines. Coroutines will just interact with parts of my program, no IO is involved. My C scheduler is single-threaded and I need coroutines to execute in its thread.
In a pure Python program, I would just use asyncio
as is, and let my program use its event loop to drive all events. This is however not an option; my event loop needs to serve millions of C-based events per second and I cannot afford Python's overheads.
I tried to write my own event loop implementation, that delegates all scheduling to my C scheduler. I tried a few approaches:
EventLoop
, Future
, Task
etc to imitate how asyncio
works (minus IO), so that call_soon
delegates scheduling to my C event loop. This is safe, but requires a bit of work and my implementation will always be inferior to asyncio
when it comes to documentation, debugging support, intricate semantic details, and correctness/test coverage.Task
, Future
etc from asyncio
, and only create a custom implementation of AbstractEventLoop
, delegating scheduling to my C event loop in the same way. This is pretty straightforward, but I can see that the vanilla EventLoop
accesses non-obvious internals (task._source_traceback
, _asyncgen_finalizer_hook
, _set_running_loop
), so my implementation is still second class. I also have to rely on the undocumented Handle._run
to actually invoke callbacks.BaseEventLoop
instead of AbstractEventLoop
(but docs say I shouldn't do that). I still need Handle._run
, though.run_forever
:s a vanilla asyncio.DefaultEventLoop
and run all my coroutines there, but coroutines depend on my program's extension API, which does not support concurrent calls. So I must somehow make DefaultEventLoop
pause my C event loop while calling Handle._run()
. I don't see a reasonable way to achieve that.Any ideas on how to best do this? How did others solve this problem?
I found that trio
, a third-party alternative to asyncio
, provides explicit support for integration with alien event loops through something called guest mode. Solves my problem!