I'm trying to use node to create a tool that will run Jasmine via karma several times based on configuration info I send it.
I'm starting with calling karma once and I'm having trouble with the server Promise resolving without the tests having run even when I'm running with singleRun: true.
If I'm going to call Jasmine several times in what the end user sees as a "single" operation, I need to compile all the return values from the function doneCallback(exitCode) lines to see if any are non-0.
That is, when singleRun: true is used, I don't think there's a chance of setting up an event listener like the docs demonstrate:
runner.run({port: 9876}).on('progress', function(data) { process.stdout.write(data) })
I'm not creating a runner and running it. I'm simply executing server.start and watching it run the tests in the config once, and I need to get the results of that. Instead, all the Promises are returning when the config is read and the server starts. No Promises are coming back to the calling context when the "implicit" runner is done during singleRun. It's fire and forget [about listening to events].
Here's a modified version of my code:
function startKarma(karmaConfig) {
var karmaPromise = karma.config
.parseConfig(
null,
karmaConfig,
// http://karma-runner.github.io/6.3/dev/public-api.html
{ promiseConfig: true, throwErrors: true }
)
.then(
(parsedKarmaConfig) => {
// http://karma-runner.github.io/6.4/dev/public-api.html
const server = new karma.Server(parsedKarmaConfig, function doneCallback(
exitCode
) {
console.log("Karma has exited with " + exitCode); // <<<<<<< happens last. exitCode is defined.
return exitCode;
});
var serverPromise = server.start();
return serverPromise.then(function (x) {
console.log("server promise", x); // <<<<<<<<<<<< happens first, x === undefined
return x;
});
},
(rejectReason) => {
console.log("Error", rejectReason);
throw rejectReason;
}
);
return karmaPromise;
}
And I kick off the code like this:
// leaving out config creation
startKarma(config).then(function (after) {
console.log("after", after); // <<<<<<<<<<<< happens second, after === undefined
});
And here's my "base config", which has files, preprocessors, and basePath overridden elsewhere in the code.
var baseConfig = {
files: [{ pattern: "**/*.js" }],
preprocessors: {
"**/!(*test).js": ["coverage"],
},
basePath: "./",
frameworks: ["jasmine"],
exclude: ["**/node_modules/**/"],
reporters: ["coverage", "mocha"],
coverageReporter: {
reporters: [
{ type: "text-summary" },
{ type: "html", dir: "../coverage/" },
{ type: "text", dir: "coverage/", file: "coverage.txt" },
],
},
port: 9876,
colors: true,
logLevel: karma.config.LOG_DEBUG,
autoWatch: true,
browsers: ["Chrome"],
singleRun: true,
concurrency: Infinity,
};
The best way I can explain what I want to do is, "I want to get a handle to the runner within the singleRun Server.start and return its results to the calling code via a Promise." Not sure where I'm supposed to access the "implicit" runner's results.
In other words, with singleRun: true, I'd expect the Server.start Promise to be returned only once the tests have been run, not as soon as the server is up and running. Since it doesn't, I'm wondering where I can hook into the runner that must be running.
Results:
START:
server promise undefined
after undefined
01 05 2023 16:27:56.094:INFO [karma-server]: Karma v6.4.2 server started at http://localhost:9876/
01 05 2023 16:27:56.097:INFO [launcher]: Launching browsers Chrome with concurrency unlimited
01 05 2023 16:27:56.103:INFO [launcher]: Starting browser Chrome
01 05 2023 16:27:56.916:INFO [Chrome 112.0.0.0 (Windows 10)]: Connected on socket fLZkvHywy1zV36iIAAAB with id 22356421
these are not real tests
add2 function in global scope
√ should return a value that is 2 greater than that which was pushed in
add2broken function in global scope
× should return a value that is 2 greater than that which was pushed in
double function in global scope
√ should return a value double that pushed in
square function in global scope
√ should return a value that squares that which was pushed in
Finished in 0.018 secs / 0.002 secs @ 16:27:56 GMT-0400 (Eastern Daylight Time)
SUMMARY:
√ 3 tests completed
× 1 test failed
FAILED TESTS:
these are not real tests
add2broken function in global scope
× should return a value that is 2 greater than that which was pushed in
Chrome 112.0.0.0 (Windows 10)
Expected 27 to be 7.
at <Jasmine>
at UserContext.<anonymous> (fakeTests/testSubdir/add2.test.js:26:28)
at <Jasmine>
=============================== Coverage summary ===============================
Statements : 76.47% ( 13/17 )
Branches : 50% ( 8/16 )
Functions : 80% ( 4/5 )
Lines : 81.25% ( 13/16 )
================================================================================
To be clear, I think setting singleRun: false and handling the server lifecycle "manually" would allow me to do what I want, but I would like to KISS and hook into a singleRun: true version if possible.
The Right Thing to do, I believe, is to wrap all that logic into my own Promise. That is, karma does not provide event hooks for single runs. That approach (described in the original question) is wasted effort.
To roll your own Promise wrapper, do this (the key being return new Promise(function (resolve, reject) //...):
function startKarma(karmaRunId, overrides) {
return new Promise(function (resolve, reject) {
overrides = Object.assign(
{},
karmaConfigTools.overridesForMochaTestingRun,
overrides
);
var karmaConfig = karmaConfigTools.createKarmaConfig(overrides);
karma.config
.parseConfig(
null,
karmaConfig,
// In most cases, parseOptions.throwErrors = true should also be set.
// This disables process exiting and allows errors to result in rejected promises.
// http://karma-runner.github.io/6.3/dev/public-api.html
{ promiseConfig: true, throwErrors: true }
)
.then(
(parsedKarmaConfig) => {
// fwiw
// http://karma-runner.github.io/6.4/dev/public-api.html
// I'm not sure why it names the callback function doneCallback.
const server = new karma.Server(
parsedKarmaConfig,
function doneCallback(exitCode) {
// 0 is success/no test failure.
// Anything else is bad. Usually 1 afaict.
utils.debugLog("Wrapped karma has exited with " + exitCode);
karmaRunResults[karmaRunId] = exitCode;
resolve(exitCode);
}
).on("progress", function (data) {
process.stdout.write(data);
});
server.start().then(function (x) {
utils.debugLog("server started", x);
});
},
(rejectReason) => {
utils.debugLog("Error", rejectReason);
throw rejectReason;
}
);
});
}
Now I can wait until I get to the section I wanted initially:
const server = new karma.Server(parsedKarmaConfig, function doneCallback(
exitCode
) {
console.log("Karma has exited with " + exitCode); // <<<<<<< happens last. exitCode is defined.
return exitCode;
});
and instead of listening for that via karma, the Promise I added, well, acts amazingly 😏 like a Promise, allowing the caller to wait around until it gets the information it wanted. If I need to call a ton of singleRun test suites, I group the calls in an array and Promise.all them. Again, typical Promise stuff.
The bottom line is that karma does not appear to allow you to listen to events from single runs. You have to set up an open-ended-lived (!singleRun) karma runner if you want to listen via karma's handler with code like runner.run({port: 9876}).on('progress', function(data) { // ...
You can, however, set up your own async (Promise-based) management with "multiple singleRun" based logic to accomplish the same thing, as shown above.