I have a RuleBasedStateMachine
which took about 40 seconds to run before. We then had a change that increased the amount of time taken in one of the steps and introduced a bug.
Because of the performance penalty, it would normally take 90 seconds to succeed but if you were unlucky it could take more than 5 minutes. If you then managed to trigger the bug, the test would shrink for more than an hour.
We managed to figure out what was wrong with @hypothesis.settings(verbosity=Verbosity.debug)
, but it took time to pinpoint which test was failing because our test runs would timeout after an hour.
Is there any way of setting a timelimit on the amount of time spent shrinking?
This is very strange, because Hypothesis already limits shrinking to five minutes by default. What version are you using?
(and to answer your exact question: there's no supported way to limit shrinking time, but if you don't mind messing with unstable internals you can patch that constant)