How can I test that Python package entrypoints are correctly discovered and loaded?

I am currently writing a package that seeks to load entrypoint contributions from other installed packages using importlib.metadata.entry_points, but am unsure how I can test this properly. I am using Pytest for my package's testing.

I'm essentially trying to solve the opposite problem of this post, where they want to ensure that their package is loaded by another installed package.

I cannot simply have example plugin packages as development dependencies, as I want to be able to test different cases with different plugins:

What happens if a plugin fails to import?
What happens if a plugin raises an exception?
Does a correctly-written plugin behave well with my application?

I want to test each of these cases in different tests, and so I need a way of dynamically registering and unregistering mock plugins for Python's entrypoint system.

Here's what my ideal code would look like:

def test_bad_plugin_class():
    class MyBadPlugin(MyPluginInterface):
        def do_something(self):
            # This code is broken
            raise RuntimeError("No.")

    with importlib.metadata.add_mock_entrypoint(  # function does not exist :(
        'my_package.plugins', 
        'bad_plugin_lib:MyBadPlugin', 
        # Give the class that should be returned when calling EntryPoint.load
        # to avoid needing to create a file to import from
        MyBadPlugin,
    ):
        # Would check for actual error handling here
        with pytest.raises(ImportError):
            my_library.load_plugins()

I could just mock the entire entrypoint system, but this feels excessive and tedious, and so I am looking for a better solution.

How can I accomplish this?

Solution

It is possible to achieve this by mocking importlib.metadata, which is much simpler to accomplish than I originally anticipated.

An example of how to accomplish this can be seen in the Poetry project's tests/helpers.py file.

Here is a documented version:

from importlib import metadata
from typing import Any
from pytest_mock import MockerFixture

def make_entry_point_from_plugin(
    name: str, cls: type[Any], dist: metadata.Distribution | None = None
) -> metadata.EntryPoint:
    """
    Create and return an importlib.metadata.EntryPoint object for the given
    plugin class.
    """
    group: str | None = getattr(cls, "group", None)
    ep = metadata.EntryPoint(
        name=name,
        group=group,  # type: ignore[arg-type]
        value=f"{cls.__module__}:{cls.__name__}",
    )

    # I am not entirely sure what's going on here, but if someone knows,
    # a comment or edit would be appreciated
    if dist:
        ep = ep._for(dist)  # type: ignore[attr-defined,no-untyped-call]
        return ep

    return ep


def mock_metadata_entry_points(
    mocker: MockerFixture,
    cls: type[Any],
    name: str = "my-plugin",
    dist: metadata.Distribution | None = None,
) -> None:
    """
    Add a mock entry-point to importlib's metadata.

    The entry-point's group is determined using the static `group` attribute of
    the given `cls`.

    Args:
        mocker (MockerFixture): the mocker to use when mocking the values.
        cls (type[Any]): the class to register as a plugin. This class must be
        defined in an importable location (eg the global scope of a valid
        module).
        name (str, optional): name to use for the plugin. Defaults to
        `"my-plugin"`.
        dist (metadata.Distribution, optional): distribution info for the
        plugin. Defaults to `None`.
    """
    mocker.patch.object(
        metadata,
        "entry_points",
        return_value=[make_entry_point_from_plugin(name, cls, dist)],
    )


# This code originates from Poetry's code for testing their own plugin system.
# https://github.com/python-poetry/poetry/blob/ecc2697fd79bbc7ef3037ea95c7ed1ef83b8a658/tests/helpers.py#L253
#
# Copyright (c) 2018-present Sébastien Eustace
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

You can use it by passing a mock class to it during a test case:

class FailToLoadPlugin:
    """Plugin that fails to load due to an exception during creation"""

    # Need to set this value so the plugin is registered correctly
    group = "my_package.plugins"

    def __init__(self) -> None:
        raise RuntimeError("Intentional failure to load plugin")

def test_handler_load_failure(mocker: MockerFixture):
    # This registers the plugin
    mock_metadata_entry_points(mocker, FailToLoadPlugin)
    with pytest.raises(PluginLoadError):
        my_library.load_plugins()