Apologies if this is somewhat vague (and it is), but I've inherited a bit of a mess from a departed contractor, and need some pointers...
I've got (inherited) a number of JavaScript Unit tests (approx. 770+). These are being run with Chutzpah (both in Visual Studio 2015, and from the command line during our Team City build).
The problem is, each time I run the tests - either in VS, or from the command line - the total number of tests run differs, as does the number of failures
It's the inconsistency that's the problem. I can't start investigating actual problems, as I don't even know if I can trust the tests that have been run.
Apologies, I appreciate I've not given any real detail here, but can anyone suggest any high level things I should look into in this sort of scenario ?
Cheers
As the commentor mentioned, you need to figure out what the flakey tests are. I would set up a batch file to run the set of tests hundreds of times and then analyze the results. Once you have a suspect you can run the tests with the /trace command line argument that creates a log file named chutzpah.log that shows all that the test did. Sometimes this can reveal differences from normal execution.
In addition a common cause of flakey tests is mutable shared state and async methods. Once you have a candidate test I would look for those patterns.