pythonpycharmpython-typing

How do I type hint a method with the type of the enclosing class?


I have the following code in Python 3:

class Position:

    def __init__(self, x: int, y: int):
        self.x = x
        self.y = y

    def __add__(self, other: Position) -> Position:
        return Position(self.x + other.x, self.y + other.y)

But my editor (PyCharm) says that the reference Position can not be resolved (in the __add__ method). How should I specify that I expect the return type to be of type Position?

I think this is actually a PyCharm issue. It actually uses the information in its warnings, and code completion.

But correct me if I'm wrong, and need to use some other syntax.


Solution

  • I guess you got this exception:

    NameError: name 'Position' is not defined
    

    This is because in the original implementation of annotations, Position must be defined before you can use it in an annotation.

    Python 3.14+: It'll just work

    Python 3.14 has a new, lazily evaluated annotation implementation specified by PEP 749 and 649. Annotations will be compiled to special __annotate__ functions, executed when an object's __annotations__ dict is first accessed instead of at the point where the annotation itself occurs.

    Thus, annotating your function as def __add__(self, other: Position) -> Position: no longer requires Position to already exist:

    class Position:
        def __add__(self, other: Position) -> Position:
            ...
    

    Python 3.7+, deprecated: from __future__ import annotations

    from __future__ import annotations turns on an older solution to this problem, PEP 563, where all annotations are saved as strings instead of as __annotate__ functions or evaluated values. This was originally planned to become the default behavior, and almost became the default in 3.10 before being reverted.

    With the acceptance of PEP 749, this will be deprecated in Python 3.14, and it will be removed in a future Python version. Still, it works for now:

    from __future__ import annotations
    
    class Position:
        def __add__(self, other: Position) -> Position:
            ...
    

    Python 3+: Use a string

    This is the original workaround, specified in PEP 484. Write your annotations as string literals containing the text of whatever expression you originally wanted to use as an annotation:

    class Position:
        def __add__(self, other: 'Position') -> 'Position':
            ...
    

    from __future__ import annotations effectively automates doing this for all annotations in a file.

    typing.Self might sometimes be appropriate

    Introduced in Python 3.11, typing.Self refers to the type of the current instance, even if that type is a subclass of the class the annotation appears in. So if you have the following code:

    from typing import Self
    
    class Parent:
        def me(self) -> Self:
            return self
    
    class Child(Parent): pass
    
    x: Child = Child().me()
    

    then Child().me() is treated as returning Child, instead of Parent.

    This isn't always what you want. But when it is, it's pretty convenient.

    For Python versions < 3.11, if you have typing_extensions installed, you can use:

    from typing_extensions import Self
    

    Sources

    The relevant parts of PEP 484, PEP 563, and PEP 649, to spare you the trip:

    Forward references

    When a type hint contains names that have not been defined yet, that definition may be expressed as a string literal, to be resolved later.

    A situation where this occurs commonly is the definition of a container class, where the class being defined occurs in the signature of some of the methods. For example, the following code (the start of a simple binary tree implementation) does not work:

    class Tree:
        def __init__(self, left: Tree, right: Tree):
            self.left = left
            self.right = right
    

    To address this, we write:

    class Tree:
        def __init__(self, left: 'Tree', right: 'Tree'):
            self.left = left
            self.right = right
    

    The string literal should contain a valid Python expression (i.e., compile(lit, '', 'eval') should be a valid code object) and it should evaluate without errors once the module has been fully loaded. The local and global namespace in which it is evaluated should be the same namespaces in which default arguments to the same function would be evaluated.

    and PEP 563, deprecated:

    Implementation

    In Python 3.10, function and variable annotations will no longer be evaluated at definition time. Instead, a string form will be preserved in the respective __annotations__ dictionary. Static type checkers will see no difference in behavior, whereas tools using annotations at runtime will have to perform postponed evaluation.

    ...

    Enabling the future behavior in Python 3.7

    The functionality described above can be enabled starting from Python 3.7 using the following special import:

    from __future__ import annotations
    

    and PEP 649:

    Overview

    This PEP adds a new dunder attribute to the objects that support annotations–functions, classes, and modules. The new attribute is called __annotate__, and is a reference to a function which computes and returns that object’s annotations dict.

    At compile time, if the definition of an object includes annotations, the Python compiler will write the expressions computing the annotations into its own function. When run, the function will return the annotations dict. The Python compiler then stores a reference to this function in __annotate__ on the object.

    Furthermore, __annotations__ is redefined to be a “data descriptor” which calls this annotation function once and caches the result.

    Things that you may be tempted to do instead

    A. Define a dummy Position

    Before the class definition, place a dummy definition:

    class Position(object):
        pass
    
    class Position:
    
        def __init__(self, x: int, y: int):
            self.x = x
            self.y = y
    
        def __add__(self, other: Position) -> Position:
            return Position(self.x + other.x, self.y + other.y)
    

    This will get rid of the NameError and may even look OK:

    >>> Position.__add__.__annotations__
    {'other': __main__.Position, 'return': __main__.Position}
    

    But is it?

    >>> for k, v in Position.__add__.__annotations__.items():
    ...     print(k, 'is Position:', v is Position)                                                                                                                                                                                                                  
    return is Position: False
    other is Position: False
    

    And mypy will report a pile of errors:

    main.py:4: error: Name "Position" already defined on line 1  [no-redef]
    main.py:11: error: Too many arguments for "Position"  [call-arg]
    main.py:11: error: "Position" has no attribute "x"  [attr-defined]
    main.py:11: error: "Position" has no attribute "y"  [attr-defined]
    Found 4 errors in 1 file (checked 1 source file)
    

    B. Monkey-patch in order to add the annotations:

    You may want to try some Python metaprogramming magic and write a decorator to monkey-patch the class definition in order to add annotations:

    class Position:
        ...
        def __add__(self, other):
            return self.__class__(self.x + other.x, self.y + other.y)
    

    The decorator should be responsible for the equivalent of this:

    Position.__add__.__annotations__['return'] = Position
    Position.__add__.__annotations__['other'] = Position
    

    It'll work right at runtime:

    >>> for k, v in Position.__add__.__annotations__.items():
    ...     print(k, 'is Position:', v is Position)                                                                                                                                                                                                                  
    return is Position: True
    other is Position: True
    

    But static analyzers like mypy won't understand it, and static analysis is the biggest use case of type annotations.