In my test code, my doctest fails but the script exits with a zero return value, which causes the CI run to pass, which is not intended.
Is this the correct behavior of doctest module?
My script ends with:
if __name__ == '__main__':
import doctest
doctest.testmod()
The output is like:
**********************************************************************
File "test/test.py", line 7, in __main__
Failed example:
f(1,0)
Expected:
-----
type: <type 'exceptions.ZeroDivisionError'>
value: integer division or modulo by zero
x
-----
Got:
-----
type: <type 'exceptions.ZeroDivisionError'>
value: integer division or modulo by zero
-----
**********************************************************************
1 items had failures:
1 of 1 in __main__
***Test Failed*** 1 failures.
tux@iPad:~/programming/exception-notifier(fix-travis)(0)$ echo $?
0
I find using doctest.testmod(raise_on_error=True)
will cause an exception to be raised when a test fails, which causes the script exits with a non-zero code.
Optional argument
raise_on_error
defaults to false. If true, an exception is raised upon the first failure or unexpected exception in an example. This allows failures to be post-mortem debugged. Default behavior is to continue running examples.