How can I programmatically capture the output from the current cell in a function called from Jupyter / ipython?

I am trying to implement an ipython's magic cell function that takes the code from the current cell, runs it, and caches its output (in rich format) to a file. The final objective of my magic function is a bit more elaborate but I hope this description suffices.

In order to achieve this, I want my magic function to use a helper function which I call capture_output, implemented in some python module, example.py, which allows me to capture the output from my current cell in a jupyter notebook, something like this:

>>> import example as ex

>>> cell_code = "3+2"

>>> output = ex.capture_output (cell_code)

>>> output.show()

5

In this case, the captured output is just text, i.e., the result of 3+2. However, if the cell displays an image, then the captured output would be the displayed image:

>>> import example as ex

>>> cell_code = """
import matplolib.pyplot as plt

plt.plot ([1,2,3,4])
"""

>>> output = ex.capture_output (cell_code)

>>> output.show()

I know we have the magic %%capture, and we can run it using "run_cell_magic". However, this produces errors when implemented inside a python module that is being called from an ipython session / Jupyter notebook.

For example, if I have the following code in "example.py":

from IPython import get_ipython

def capture_output (code):
    get_ipython().run_cell_magic("capture", "output", code)
    return output

And then run it from an ipython session:

import example as ex

output = ex.capture_output ("3+2")
output.show()

I get the following error:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 ex.capture_output ("3+2")

File example.py:6, in capture_output (code)
      4 def capture_output (code):
      5     get_ipython().run_cell_magic("capture", "output", code)
----> 6     return output

NameError: name 'output' is not defined

Solution

You want the following for example.py

from IPython import get_ipython

def capture_output (code):
    get_ipython().run_cell_magic("capture", "output", code)
    return get_ipython().user_ns['output']

Details

You put example.py as the following:

from IPython import get_ipython

def capture_output (code):
    get_ipython().run_cell_magic("capture", "output", code)
    return output

The issue is that you were expecting output to be something and you returned it; however, as written, it wasn't anything. Or at least not accessible to you. The output from the cell magic run was only accessible in the namespace of the where that ipython code was run, which is not the main namespace where you are running your code that is passing in code and expecting it returned.
You can though access IPython's user namespace as user_ns if you add to your code.

This example in response to 'How to make caller's namespace available to IPython's magic inside imported function?' using a line magic works and get me thinking along the lines of user_ns. However, the cited resource was no longer available. The answers to 'IPython notebook: How to write cell magic which can access notebook variables?' also brought up user_ns, but it wasn't obvious. Luckily, the post 'Let's Write an IPython Extension the Hard Way' by Sebastian Witowski pointed me in the correct direction & made me think ipython.user_ns may be the same as get_ipython().user_ns.

Related resources for those ending up in this thread: