Testing script output

I have a Python script that accepts one input (a text file): ./myprog.py file.txt. The script outputs a string based on the given input.

I have a set of test files that I would like to test my program with. I know the expected output for each file and want to make sure my script produces the correct output for each file.

What is the generally accepted way to do this type of testing?

I was thinking of using Python's unittest module as the testing framework, and then running my script through subprocess.check_output(stderr=subprocess.STDOUT), capturing stdout and stderr, and then doing a unittest assertEqual to compare the actual and expected strings. I want to make sure I'm not missing some nicer solution.

There's two problems here. Testing a program, as opposed to a library of functions, and testing something that prints, as opposed to values returned from a function. Both make testing more difficult, so it's best to side step these problems as much as possible.

The usual technique is to create a library of functions and then have your program be a thin wrapper around that. These functions return their results, and only the program does the printing. This means you can use normal unit testing techniques for most of the code.

You can have a single file which is both a library and a program. Here's a simple example as hello.py.

def hello(greeting, place):
    return greeting + ", " + place + "!"

def main():
    print(hello("Hello", "World"))

if __name__ == '__main__':
    main()

That last bit is how a file can tell if it was run as a program or if it was imported as a library. It allows access to the individual functions with import hello, and it also allows the file to run as a program. See this answer for more information.

Then you can write a mostly normal unit test.

import hello
import unittest
import sys
from StringIO import StringIO
import subprocess

class TestHello(unittest.TestCase):
    def test_hello(self):
        self.assertEqual(
             hello.hello("foo", "bar"),
            "foo, bar!"
        )

    def test_main(self):
        saved_stdout = sys.stdout
        try:
            out = StringIO()
            sys.stdout = out
            hello.main()
            output = out.getvalue()
            self.assertEqual(output, "Hello, World!\n")
        finally:
            sys.stdout = saved_stdout

    def test_as_program(self):
        self.assertEqual(
            subprocess.check_output(["python", "hello.py"]),
            "Hello, World!\n"
        )

if __name__ == '__main__':
    unittest.main()

Here test_hello is unit testing hello directly as a function; and in a more complicated program there would be more functions to test. We also have test_main to unit test main using StringIO to capture its output. Finally, we ensure the program will run as a program with test_as_program.

The important thing is to test as much of the functionality as functions returning data, and to test as little as possible as printed and formatted strings, and almost nothing via running the program itself. By the time we're actually testing the program, all we need to do is check that it calls main.