Generator objects in Python are required to have a close
method that exists to ensure that context managers are exited and try...finally:
blocks are run before the object is garbage collected.
PEP 342 defines the methods send
,throw
,close
and __del__
that generators must implement. Specifically, it states:
g.__del__()
is a wrapper forg.close()
. This will be called when the generator object is garbage-collected (in CPython, this is when its reference count goes to zero).
The abstract type for Generator is in collections.abc
class Generator(Iterator):
__slots__ = ()
def __next__(self):
"""Return the next item from the generator.
When exhausted, raise StopIteration.
"""
return self.send(None)
@abstractmethod
def send(self, value):
"""Send a value into the generator.
Return next yielded value or raise StopIteration.
"""
raise StopIteration
@abstractmethod
def throw(self, typ, val=None, tb=None):
"""Raise an exception in the generator.
Return next yielded value or raise StopIteration.
"""
if val is None:
if tb is None:
raise typ
val = typ()
if tb is not None:
val = val.with_traceback(tb)
raise val
def close(self):
"""Raise GeneratorExit inside generator.
"""
try:
self.throw(GeneratorExit)
except (GeneratorExit, StopIteration):
pass
else:
raise RuntimeError("generator ignored GeneratorExit")
@classmethod
def __subclasshook__(cls, C):
if cls is Generator:
return _check_methods(C, '__iter__', '__next__',
'send', 'throw', 'close')
return NotImplemented
This abstract type enforces that send
, throw
, and close
are implemented in subclasses, but doesn't implement __del__
either abstractly or concretely, or enforce that it's implemented. Its metaclasses don't either.
Naively, producing a subclass that doesn't manually define a __del__
which wraps close
gives Generators which are not correctly cleaned up after. The garbage collector only calls __del__
, so if __del__
doesn't exist, close
is not called.
Is this intentional?
In a related question, snakecharmerb pointed out to me that __del__
can be fraught to implement, as indicated by the language reference, but I can't understand why that wouldn't also apply to the correct implementation of __del__
as a wrapper for close
in Python's native generator objects.
It seems to be intentional. In an issue on the python bugtracker discussing this, Guido said that
The presence of a
__del__
method can cause subtle behavior changes to the GC, so I worry that adding__del__
to that class now is going to break currently-working code.Let's not destabilize the Generator class.