documentationpython-sphinxautodocpython-3.12

Sphinx - autodoc imported data with own docstring


The situation

I want to use the Sphinx autodoc features to document a Python API that is implemented in the following way. I have a module that is used to expose various symbols (functions, constants, type synonyms, classes) that are mostly implemented elsewhere. Those symbols are therefore imported from private modules with the classical syntax from _privateModule import a,b,c. In the modules where those symbols are implemented I have prepared docstrings for all of them. I think this is kind of a usual way of putting together APIs.

The problem

Sphinx autodoc functionality seems to be unable to grab the symbols with their docstrings in some situations, e.g. with type aliases. It does work well with classes and functions, though. There seems to be no way of having the type aliases autodoc-ed as part of the main API module, and have them show their docstring at the same time. For instance, in the example that I added below, the Mydata parameter shows up in the docs, but its description is auto generated; the dosctring that I wrote is not picked up by Sphinx.

Also, please consider that due to the size of my API, moving the docstrings away from the implementer modules is not an option.

What I tried so far

I have tried several approaches, including:

Here below a minimum working example:

Sphinx Markdown syntax (I use myst):

# My API

Sample API docs.

## Some stuff

Blah blah

```{eval-rst}
.. automodule:: example
   :members:
   :imported-members:
   :undoc-members:
.. autodata:: Mydata

Python exposition file, example.py:

"""
This module implements an API by defining some stuff here and importing other
stuff from another private module.
"""

from _aux import (
    myfun,
    Mydata,
    Myclass,
)

class ExampleClass:
    """'ExampleClass' is implemented in `example.py`."""

    example_member: float

__all__ = [
    'myfun',
    'Mydata',
    'Myclass',
    'ExampleClass',
]

Python implementer file _aux.py:

def myfun(x):
    """Myfun is implemented in `_aux.py` and imported into `example.py`."""
    return 2*x


Mydata = list[str]
"""
Mydata is implemented in `_aux.py` and imported into `example.py`.
"""


class Myclass():
    """
    Myclass is implemented in `_aux.py` and imported into `example.py`.
    """

    a_member: int
    another_member: float

Solution

  • This is apparently a known Sphinx limitation.

    The reason it works for classes and functions, but not for other things, has to do with the way Python represents docstrings. For classes and functions, Sphinx can read your docstring by importing your package and reading public_top_level_module.foo.__doc__. Not so with plain assignments like data constants and type aliases, which have no __doc__ attribute.

    Sphinx tries to work around that by implementing its own parsing to find docstrings adjacent to assignments. Unfortunately, it's not smart enough to follow re-exports, and it's unclear to me whether that could even work in theory—how would it identify the module that the value originates from?


    There don't seem to be any great workaround options, unfortunately.

    For some objects, you can do:

    Foo = ...
    Foo.__doc__ = """blah blah"""
    

    But that won't work in the case of Foo being a type alias like Foo = list[str]. Trying to set Foo.___doc__ will raise an error at best, or overwrite the underlying type's docstring at worst.

    You could give up on autodoc for type aliases and manually add py:type directives to the docstring of your public module. But as you mentioned, this is difficult to maintain for large APIs.

    PEP 727 might provide another workaround if it gets accepted, but the Sphinx maintainers don't seem thrilled about it, so I wouldn't hold your breath.