nbconvert CLI: Execution until first error?

I want to run an ipynb via CLI, halt execution at the first error, and save the resulting file (including the error message). The resulting file should contain the output up to the failing cell.

Unfortunately, jupyter nbconvert --execute [myfile.ipynb] doesn't work, because it doesn't write any file when there's an error. Adding --allow-errors doesn't satisfy my requirements, because it does not halt after the first error.

The docs show how to achieve this in code ("Execution until first error"):

An error during the notebook execution, by default, will stop the execution and raise a CellExecutionError. Conveniently, the source cell causing the error and the original error name and message are also printed. After an error, we can still save the notebook as before:

with open('executed_notebook.ipynb', mode='w', encoding='utf-8') as f:
   nbformat.write(nb, f)

Can it be done via CLI?

It works on the command line if you use Papermill.

Example Papermill Use:

I set up a demo notebook with one code cell with good code and the next code cell having a syntax error, followed by other code cells with good code.

Here is the record of using Papermill to execute that notebook to test error handling:

jovyan@jupyter-binder-2dexamples-2drequirements-2dxhpwwlxe:~$ papermill test_error_handling.ipynb run_with_papermill.ipynb
Input Notebook:  test_error_handling.ipynb
Output Notebook: run_with_papermill.ipynb
Executing:   0%|                                                                                                              | 0/5 [00:00<?, ?cell/s]Executing notebook with kernel: python3
Executing:  60%|█████████████████████████████████████████████████████████████▏                                        | 3/5 [00:02<00:01,  1.13cell/s]
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/bin/papermill", line 8, in <module>
    sys.exit(papermill())
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/papermill/cli.py", line 235, in papermill
    execute_notebook(
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/papermill/execute.py", line 131, in execute_notebook
    raise_for_execution_errors(nb, output_path)
  File "/srv/conda/envs/notebook/lib/python3.10/site-packages/papermill/execute.py", line 251, in raise_for_execution_errors
    raise error
papermill.exceptions.PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [2]":
  Cell In [2], line 1
    print(
          ^
SyntaxError: incomplete input

Already the command line progress shows you it doesn't complete it 100% before ceasing. It does though save the progress in the output .ipynb file though.

The resulting .ipynb file produced by that demo run is here.
Examining the produced file, you'll see it stops at the 2nd cell and does not continue to run any of the cells below the point the error occurs. The error it encountered is built into the notebook output and also reported in the command line interface where running it was triggered.

Papermill though does add notification about the error at the top and the point just above the error. The markdown content that Papermill adds both have the text papermill-error-cell included in them and so you can easily design a script using nbformat to clean up this cruft if you prefer.

Jupytext was also tried

I also tried Jupytext via CLI and it doesn't save the file if it hits an error, as detailed here in 'Using CLI: Advanced usage: error tolerance'.

Related:

• discussion in relation to 'Auto-save periodically during nbconvert'
• discussion in relation to 'Run .ipynb on a remote workstation so that all results will be save in the same .pynb'
• method to save every second using api, which is linked from discussion in relation 'Auto-save periodically during nbconvert ' using api