pythonoverridingpython-decoratorspython-typingpyright

How to type annotate a decorator to make it Override-friendly?


I find that adding type annotation for a decorator will cause typing error in overriding case in VSCode Pylance strict mode. following is a minimal-complete-example (in Python3.8 or later?):

from datetime import datetime
from functools import wraps
from typing import Any

_persistent_file = open("./persistent.log", "a")

def to_json(obj: Any) -> str:
    ...

def persistent(class_method: ...):
    @wraps(class_method)
    async def _class_method(ref: Any, *args: Any):
        rst = await class_method(ref, *args)
        print(f'{ datetime.now().strftime("%Y-%m-%d %H:%M:%S") } { to_json(args) } { to_json(rst) }', file=_persistent_file)
        return rst
    return _class_method

class A:
    async def f(self, x: int):
        return x

class B (A):
    @persistent
    async def f(self, x: int):
        return x + 1

Here, B.f will be marked as type error in pylance:

"f" overrides method of same name in class "A" with incompatible type "_Wrapped[(...), Any, (ref: Any, *args: Any), Coroutine[Any, Any, ...]]"

Pylance (reportIncompatibleMethodOverride)

And I find that remove @wraps will workaround this issue, but I need it to keep the metadata of the function (e.g. __name__).

Hoping for a perfect solution.


Solution

  • For highly generic instance method decorators, I would typically do something like this: (I hope it is OK that I drastically simplified your example)

    from collections.abc import Callable, Coroutine
    from functools import wraps
    from typing import Any, TypeVar
    from typing_extensions import Concatenate, ParamSpec
    
    P = ParamSpec("P")
    R = TypeVar("R")
    S = TypeVar("S")
    
    
    def decorator(
        class_method: Callable[Concatenate[S, P], Coroutine[Any, Any, R]]
    ) -> Callable[Concatenate[S, P], Coroutine[Any, Any, R]]:
        @wraps(class_method)
        async def _class_method(self: S, /, *args: P.args, **kwargs: P.kwargs) -> R:
            rst = await class_method(self, *args, **kwargs)
            print("...")
            return rst
        return _class_method
    
    
    class A:
        async def f(self, x: int) -> int:
            return x
    
    
    class B(A):
        @decorator
        async def f(self, x: int) -> int:
            return x + 1
    

    Here S represents the instance type (self), R stands for the return type and P for the parameters of the decorated method. With Python >=3.10 you can import Concatenate and ParamSpec directly from typing.

    Note that making the first argument (self) to the wrapper method positional-only by placing it before a / in the parameters is actually important because instance methods obviously bind the instance itself to the first argument, thus not ever allowing self to be passed as a keyword argument.

    I have not tested this with Python 3.8, but for >=3.9 this passes mypy --strict, so I would expect it to pass Pyright too.