[SOLVED] Jasmine tests randomly fail due to timeout when run under Chutzpah

Jasmine tests randomly fail due to timeout when run under Chutzpah

I have been struggling to get Chutzpah and Jasmine to play nicely. This is on a legacy project that's no longer actively maintained, so I'm trying to work with what I have at the moment.

When running tests with Chutzpah, I get an assortment of bogus errors:

Tests exceeding the async timeout: Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
- The vast majority of tests that throw this error are completely synchronous and don't use promises, async/await, or the done() callback.
- Re-running these exact tests will pass almost immediately with sub-100ms runtimes.
Random undefined objects: Uncaught TypeError: Cannot read properties of undefined (reading '<object>') thrown
- Only occurs in single tests within massive test suites.
- Re-running these exact tests will pass with no warnings.

The frontend of this project is built with Knockout, Durandel, and RequireJS. Below is my Chutzpah.json:

{
    "Framework": "jasmine",
    "FrameworkVersion": "2",
    "TestHarnessReferenceMode": "AMD",
    "TestHarnessLocationMode": "SettingsFileAdjacent",
    "References": [
        { "Path": "require.js" },
        { "Path": "config.js" }
    ],
    "Tests": [{ "Path": "specs" }],
    "CodeCoverageIncludes": ["*\\app\\*"],
    "CodeCoverageSuccessPercentage": 84, //currently our lowest completely tested file percentage
    "Engine": "Chrome",
    "EngineOptions": {
        "ChromeBrowserPath": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
    },
    "TestFileTimeout": 20000
}

Solution

My problem ended up being that Chutzpah could not run multiple tests at once without timing out. Set your parallelism value to 1 and your tests should stop timing out.