pythonpython-typingpyright

Polymorphism in Callablle under python type checking (Pylance)


For my code I have an aggregate class that needs a validation method defined for each of the subclasses of base class BaseC, in this case InheritC inherits from BaseC.

The validation method is then passed into the aggregate class through a register method.

See the following simple example

from typing import Callable


class BaseC:
    def __init__(self) -> None:
        pass
    
class InheritC(BaseC):
    def __init__(self) -> None:
        super().__init__()

    @classmethod
    def validate(cls, c:'InheritC') ->bool:
        return False

class AggrC:
    def register_validate_fn(self, fn: Callable[[BaseC], bool])-> None:
        self.validate_fn = fn

ac = AggrC()
ic = InheritC()
ac.validate_fn(ic.fn)

I added type hints on the parameter for registering a function, which is a Callable object Callable[[BaseC], bool] since potentially there will be several other validation methods which is defined for each class inherited from BaseC.

However, pylance doesn't seem to recognize this polymorphism in a Callable type hint and throws a warning (I set up my VScode to type check it) that said

Argument of type "(c: InheritC) -> bool" cannot be assigned to parameter "fn" of type "(BaseC) -> bool" in function "register_fn"
  Type "(c: InheritC) -> bool" cannot be assigned to type "(BaseC) -> bool"
    Parameter 1: type "BaseC" cannot be assigned to type "InheritC"
      "BaseC" is incompatible with "InheritC" Pylance(reportGeneralTypeIssues)

I don't see where I made an mistake in design, and I don't want to simply ignore the warning.

Can any one explain why this is invaid? Or is it just simply a bug from pylance

I'm using python version 3.8.13 for development.


Solution

  • I'll be using the below sample code, where I've fixed a couple of bugs:

    from typing import Callable
    
    
    class BaseC:
        def __init__(self) -> None:
            pass
    
    class InheritC(BaseC):
        def __init__(self) -> None:
            super().__init__()
    
        @classmethod
        def validate(cls, c:'InheritC') ->bool:
            return False
    
    class AggrC:
        def register_validate_fn(self, fn: Callable[[BaseC], bool])-> None:
            self.validate_fn = fn
    
    ac = AggrC()
    ic = InheritC()
    ac.register_validate_fn(ic.validate)
    

    This Python runs without errors, but still produces the same error you're seeing when run through a type checker (in my case, MyPY):

    $ mypy stackoverflow_test.py 
    stackoverflow_test.py:21: error: Argument 1 to "register_validate_fn" of "AggrC" has incompatible type "Callable[[], bool]"; expected "Callable[[BaseC], bool]"  [arg-type]
    Found 1 error in 1 file (checked 1 source file)
    

    This is a subtle issue, which is easy to overlook: it's an issue with contravariance of parameter types.

    The reason this is easy to overlook is because most object-oriented tutelage focuses on classes and objects, and doesn't really discuss functions as being types with inheritance. Indeed, most languages with object-oriented language features don't support declaring a function as inheriting from another function!

    Lets reduce this as much as possible:

    from typing import Callable
    
    class Parent:
        def foo(self):
            pass
    
    class Child(Parent):
        def bar(self):
            pass
        
    def takesParent(parameter: Parent):
        parameter.foo()
        
    def takesChild(parameter: Child):
        parameter.bar()
        
    def takesFunction(function: Callable):
        # What should the signature of `function` be to support both functions above?
        pass
    

    How should you define function: Callable to make it compatible with both functions?

    Lets take a look at what takesFunction could do, which would be valid for both functions:

    def takesFunction(function: Callable):
        child = Child()
        function(child)
    

    This function should work if you pass either function, because takesParent will call child.foo(), which is valid; and takesChild will call child.bar(), which is also valid.

    OK, how about this function?

    def takesFunction(function: Callable):
        parent = Parent()
        function(parent)
    

    In this case, function(parent) can only work with takesParent, because if you pass takesChild, takesChild will call parent.bar() - which doesn't exist!

    So, the signature that supports passing both functions looks like this:

    def takesFunction(function: Callable[[Child], None]):
    

    Which is counter-intuitive to many people.

    The parameter function must be type-hinted as taking the most specific parameter type. Passing a function with a less specific parameter - a superclass - is compatible, but passing one with a more specific parameter isn't.

    This can be a difficult topic to understand, so I'm sorry if I didn't make it very clear, but I hope this answer helped.