[SOLVED] Python: contextlib.redirect_stdout sometimes doesn't work in real time

Python: contextlib.redirect_stdout sometimes doesn't work in real time

I am using Python3.5 on Ubuntu. The following script creates a file out and fills it with more and more lines of "Hello":

import contextlib
with contextlib.redirect_stdout(open('out','w')):
    while True:
        print('Hello')

However, if I make a tiny modification by adding a call to time.sleep(), the file out stays empty:

import contextlib
import time
with contextlib.redirect_stdout(open('out','w')):
    while True:
        print('Hello')
        time.sleep(1)

If I further change the code by turning the loop into a finite loop, it fills out but only in a lump at the end of the loop.

Can anyone reproduce this and explain it?

Solution

This is caused by output buffering. This is an interesting case given the use of contextlib, but the underlying issue is with the open('out', 'w') statement.

To prevent output buffering, you can set the buffering argument to 0 if you're using Python 2:

import contextlib
import time

with contextlib.redirect_stdout(open('out','w', 0)):
    while True:
        print 'Hello'
        time.sleep(1)

Alternatively, the file contents will be written once the context manager closes the file (or once the buffered content exceeds a certain size).

If you're using Python 3, then you can't have unbuffered text I/O (kindly pointed out in the comments). However, flushing stdout at the end of each loop should have the desired effect:

import contextlib
import time
import sys

with contextlib.redirect_stdout(open('out','w')):
    while True:
        print('Hello')
        sys.stdout.flush()
        time.sleep(1)