pythonjupyterpapermill

Add papermill parameters to notebook manually

Is is possible to add papermill parameters to a jupyter notebook manually, e.g. in an editor? Is it possible to add papermill parameters to a .py file and have them persist when converted to a .pynb file?

Context:

I am working on running jupyter notebooks in an automated way via Papermill. I would like to add parameters to a notebook manually rather than using jupyter or jupyter lab interfaces. Ideally these parameters could be added to a python script .py file first. Upon converting the .py file to a .ipynb file the parameters would persist.

My desired workflow looks like this:

  1. Store generic notebook as a < notebook >.py file with parameters in version control repository
  2. Convert python script < notebook >.py to jupyter notebook < notebook >.ipynb
  3. Run < notebook >.ipynb via papermill and pass parameters to it
  4. Use nbconvert to produce output pdf with no code (using exclude_input argument)

Steps 1-3 will be run via a script that can be auotmated. I want to use jupytext to avoid storing the notebooks and all their associated metadata. Currently, the only way I can find to add parameters to a notebook is to add them via jupyter/(lab) interfaces. Then I can run the notebook with papermill. However, this doesn't work with the jupytext conversion.

*Note I would have added the "jupytext" tag to this but it doesn't exist yet and I don't have enough rep to create

EDIT

gooseberry's answer appears to be the correct one.

However, it doesn't actually appear to be necessary to add a parameters tag to your notebook in order to inject parameters via papermill. While papermill will give a no cell with tag parameters found in notebook warning it will still inject the parameters. Additionally, your output notebook from papermill will have a new cell:

    # Parameters
    parm1 = <val passed to papermill>
    parm2 = <val passed to papermill>
    etc.

versions:

Solution

  • It depends which of the formats you chose for your .py files, but assuming you've chosen the "percent" format, where each new cell is marked with #%%, to add tags you write:

    #%% tags=["parameters"]
my_text = ""
print(my_text)

    Now you'll be able to inject a different value of my_text using Papermill.

    You can find more info about formats here: https://jupytext.readthedocs.io/en/latest/formats.html