pythoninheritanceoverridingmypypython-typing

Overriding a method, mypy throws an "incompatible with super type" error when changing return type to child class type


For the following code:

class Person:
    number: int

    def __init__(self) -> None:
        self.number = 0

    def __add__(self, other: 'Person') -> 'Person':
        self.number = self.number + other.number
        return self


class Dan(Person):
    def __init__(self) -> None:
        super().__init__()

    def __add__(self, other: 'Dan') -> 'Dan':
        self.number = self.number + other.number + 1
        return self

MyPy Output:

test.py:15: error: Argument 1 of "__add__" is incompatible with supertype "Person"; supertype defines the argument type as "Person"  [override]
test.py:15: note: This violates the Liskov substitution principle
test.py:15: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides

I would think that the child type, Dan, is more specific than Person and would not violate the Liskov substitution principle.

My first thought for a solution was to create a type variable person_co = TypeVar("person_co", bound=Person). But that did not work because Person was not yet defined. I also tried making the argument and return type Person for the __add__ method in Dan. That worked and MyPy did not throw errors but that is not correct, right?


Solution

  • Why it violates the LSP

    The LSP states that method return types are covariant, whereas method parameter types are contravariant in the subtype. You shall not strengthen the preconditions, i.e. you shall not require Dan -- a strict subtype of Person -- for other in the Dan.__add__ method, if you annotated other with Person in the signature of Person.__add__.

    Look at the following function:

    def f(p: Person, q: Person) -> None:
        out = p + q
        print(out.number)
    

    Since p must be a Person, I expect that I can pass some Dan instance to it. I can also pass some Person instance to q. I would expect p + q to work because I expect I can substitute any Dan instance, where a Person instance is required.

    Your definition of the overridden method Dan.__add__ would mean that I can not pass a Dan instance for p because your method would not actually allow q to be any Person, only Dan.

    That is why your code violates the Liskov substitution principle. It is not because of the return type annotation, but because of the argument type annotation.


    Option A: Just stick to Person

    From your example it seems you have a few different options to fix this.

    The simplest one would be to keep other: Person in your overridden method:

    from __future__ import annotations
    
    
    class Person:
        number: int
    
        def __init__(self) -> None:
            self.number = 0
    
        def __add__(self, other: Person) -> Person:
            self.number = self.number + other.number
            return self
    
    
    class Dan(Person):
        def __init__(self) -> None:
            super().__init__()
    
        def __add__(self, other: Person) -> Dan:
            self.number = self.number + other.number + 1
            return self
    

    This seems reasonable since your method only relies on other having the number attribute and it being something that can be added to an integer.

    (By the way, you can omit the quotation marks, if you add from __future__ import annotations at the top of your module.)


    Option B: Type variable for genericity

    A more sophisticated option is the one you mentioned that uses a type variable, making the method generic in terms of one (or both) of its parameters. You can set the upper bound by enclosing it in quotes, if its definition comes later. (Here the __future__ import doesn't help us.)

    Something like this works:

    from __future__ import annotations
    from typing import TypeVar
    
    
    P = TypeVar("P", bound="Person")
    
    
    class Person:
        number: int
    
        def __init__(self) -> None:
            self.number = 0
    
        def __add__(self: P, other: Person) -> P:
            self.number = self.number + other.number
            return self
    
    
    class Dan(Person):
        def __init__(self) -> None:
            super().__init__()
    
        def __add__(self: P, other: Person) -> P:
            self.number = self.number + other.number + 1
            return self
    

    Now the return type will depend on what exact subtype of Person the called method is bound to. This means that even without an additional override an instance of any subclass P of Person calling __add__ would return that same type P.

    The other parameter could of course also be annotated with P, but that would result in P collapsing the the next common ancestor, if self and other are different, which in practice would mean that you could only add a subtype of Dan to an instance of Dan. That would be a much stronger requirement. I don't know, if you want that.


    Option C: Protocol for maximum flexibility

    Lastly, to show a completely different approach, you could leverage structural subtyping (introduced by PEP 544) by declaring that the only relevant aspect about other is its number and that it can be added to an int (for example because it is also an int). You could define a Protocol that stands for any type that has such an attribute. This would be a much broader definition for your method:

    from typing import Protocol, TypeVar
    
    
    P = TypeVar("P", bound="Person")
    
    
    class HasNumber(Protocol):
        number: int
    
    
    class Person:
        number: int
    
        def __init__(self) -> None:
            self.number = 0
    
        def __add__(self: P, other: HasNumber) -> P:
            self.number = self.number + other.number
            return self
    
    
    class Dan(Person):
        def __init__(self) -> None:
            super().__init__()
    
        def __add__(self: P, other: HasNumber) -> P:
            self.number = self.number + other.number + 1
            return self
    

    Note that this means that some completely unrelated class (in terms of nominal inheritance) that has the number attribute of the type int can be added to an instance of Person. From the way your example is written, I don't see any problem with that and it is arguably more in line with the dynamic philosophy of Python.


    Option D: Self for Python 3.11+

    Similarly to what SUTerliakov wrote in his answer, you could simply indicate that __add__ returns whatever type the class is.

    Combining this with the Protocol from Option C would seem to be a nice combination of simplicity (no need for a type variable and no need to annotate lowercase self) and flexibility ("if it quacks like a duck..."):

    from typing import Protocol, Self
    
    
    class HasNumber(Protocol):
        number: int
    
    
    class Person:
        number: int
    
        def __init__(self) -> None:
            self.number = 0
    
        def __add__(self, other: HasNumber) -> Self:
            self.number = self.number + other.number
            return self
    
    
    class Dan(Person):
        def __init__(self) -> None:
            super().__init__()
    
        def __add__(self, other: HasNumber) -> Self:
            self.number = self.number + other.number + 1
            return self
    

    Mix and match as you wish. As you can see, there are many options.