Passing generator expressions to any() and all()

I was just messing around in the Python interpreter and I came across some unexpected behavior.

>>> bools = (True, True, True, False)
>>> all(bools)
False
>>> any(bools)
True

Ok, so far nothing out of the ordinary...

>>> bools = (b for b in (True, True, True, False))
>>> all(bools)
False
>>> any(bools)
False

Here's where things start getting spooky. I figure this happens because the all function iterates over the generator expression, calling its __next__ method and using up the values until it encounters one that is False. Here's some evidence to back that theory up:

>>> bools = (b for b in (True, False, True, True))
>>> all(bools)
False
>>> any(bools)
True

I think the result is different because the False is not at the end, so there are still some unused values left in the generator. If you type

>>> bools = (b for b in (True, False, True, True))
>>> all(bools)
False
>>> list(bools)
[True, True]

It seems like there are only 2 remaining values.

So, why exactly does this really happen? I'm sure there are many details that I'm missing.

Solution

The behaviour of all() and any() are documented in the official documentation.

From the pseudo-code:

def all(iterable):
    for element in iterable:
        if not element:
            return False
    return True

all() only consumes True elements, it terminates when it finds the first element that evaluates to False.

def any(iterable):
    for element in iterable:
        if element:
            return True
    return False

any() consumes only False elements, it terminates when it finds the first element that evaluates to True.

Note that generators are not reset to their initial position when passed around. They stay at their current position unless more items are consumed. Hence,

>>> bools = (b for b in (True, False, True, True))

The following will consume the first two items. Since the second item is False, the iteration stops after that. This leaves the generator at a position after the second element.

>>> all(bools)
False

At this point the generator has (True, True) as the remaining values. You point that out correctly in your question. The following only consumes a single element.

>>> any(bools)
True

Note that there is still another True value obtainable from the generator after calling any().

And of course, if you call list() on a generator, all items from the generator are consumed and the generator will not yield any more items (it is "empty").