pythonasynchronouspython-asyncioipython

How to run async code in IPython startup files?

I have set IPYTHONDIR=.ipython, and created a startup file at .ipython/profile_default/startup/01_hello.py. Now, when I run ipython, it executes the contents of that file as if they had been entered into the IPython shell.

I can run sync code this way:

# contents of 01_hello.py
print( "hello!" )

$ ipython
Python 3.12.0 (main, Nov 12 2023, 10:40:37) [GCC 11.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
hello

In [1]:

I can also run async code directly in the shell:

# contents of 01_hello.py
print( "hello!" )
async def foo():
    print( "foo" )

$ ipython
Python 3.12.0 (main, Nov 12 2023, 10:40:37) [GCC 11.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
hello

In [1]: await foo()
foo

In [2]:

However, I cannot run async code in the startup file, even though it's supposed to be as if that code was entered into the shell:

# contents of 01_hello.py
print( "hello!" )
async def foo():
    print( "foo" )
await foo()

$ ipython
Python 3.12.0 (main, Nov 12 2023, 10:40:37) [GCC 11.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
[TerminalIPythonApp] WARNING | Unknown error in handling startup files:
  File ~/proj/.ipython/profile_default/startup/01_imports.py:5
    await foo()
    ^
SyntaxError: 'await' outside function

Question: Why doesn't this work, and is there a way to run async code in the startup file without explicitly starting a new event loop just for that? (asyncio.run())

Doing that wouldn't make sense, since that event loop would have to close by the end of the file, which makes it impossible to do any initialization work that involves context vars (which is where Tortoise-ORM stores its connections), which defeats the purpose.

Or stated differently: How can I access the event loop that IPython starts for the benefit of the interactive shell?

Solution

  • From version 8, ipython uses a function called get_asyncio_loop to get access to the event loop that it runs async cells on. You can use this event loop during your startup script to run any tasks you want on the same event loop that async cells will run on.

    NB. This is only uses for the asyncio package in Python's standard library and not any other async libraries (such as trio).

    from IPython.core.async_helpers import get_asyncio_loop as _get_asyncio_loop

async def foo():
    print("foo")

_get_asyncio_loop().run_until_complete(foo())

    Caveat

    The event loop that ipython uses DOES NOT run in the background. What this means is that unless you are running an async cell, no tasks that you have started will be running. ie. None of your Tortoise ORM connections will be serviced, which may cause them to break.

    As such, you may need to run your Tortoise ORM in a separate event loop anyway, and write some glue for passing data back and forth between the two event loops.