python-hypothesis

Is it possibe to put a timelimit on shrinking?


I have a RuleBasedStateMachine which took about 40 seconds to run before. We then had a change that increased the amount of time taken in one of the steps and introduced a bug.

Because of the performance penalty, it would normally take 90 seconds to succeed but if you were unlucky it could take more than 5 minutes. If you then managed to trigger the bug, the test would shrink for more than an hour.

We managed to figure out what was wrong with @hypothesis.settings(verbosity=Verbosity.debug), but it took time to pinpoint which test was failing because our test runs would timeout after an hour.

Is there any way of setting a timelimit on the amount of time spent shrinking?


Solution

  • This is very strange, because Hypothesis already limits shrinking to five minutes by default. What version are you using?

    (and to answer your exact question: there's no supported way to limit shrinking time, but if you don't mind messing with unstable internals you can patch that constant)