pythongenerator-expression

What happens to the opened file object inside a Python generator expression?


My question is, what happens to the file object created inside a generator expression after the generator's iterator is exhausted? Has close been called on it? or this benefit only comes while reading file using with statement?

Example

my_file = (nth_row for nth_row in open(file_path, "rt"))

while True:
    try:
        print(next(my_file))

    except StopIteration:
        print("End of the file")
        break


Solution

  • The close() method is not called. Generator expressions don't behave like with statements; they don't care whether you use a context manager. The code may result in a memory leak. Generally, the garbage collector will close the file, but that's not guaranteed. You could wrap the whole thing in a with block:

    with open(file_path, "rt") as file:
        my_file = (nth_row for nth_row in file)
        
        while True:
            try:
                print(next(my_file))
        
            except StopIteration:
                print("End of the file")
                break
    

    If you want to read the file in chunks, see Lazy Method for Reading Big File in Python? You can replace the .read(chunk_size) with readline() if you want.