jestjsts-jestbabel-jest

Code coverage not working as expected on Linux and Mac


I have a Next.js project which uses React, MUI and TypeScript and Jest (with RTL) for testing.

I have achieved 100% test coverage on Windows, but when I run the test coverage on a Mac or my Ubuntu CI with GitHub Actions, the test coverage is shown to be less. When I open the report, it looks very strange.

This is how the file looks like when I open it:

enter image description here

I have a dedicated spec file to test that component entirely and when ran with a separate command that executes only that spec to that component, I get the 100% test coverage.

npm test -- /Loader.spec.tsx --coverage --collectCoverageFrom=/Loader.tsx

Another component always has the code coverage issue when using only npm test (jest) whereas this one has the issue when I add the parameter --runInBand.

My Jest version in the package.json is:

"jest": "^29.7.0",
"jest-environment-jsdom": "^29.7.0",

Here are my Jest settings:

/**
 * For a detailed explanation regarding each configuration property, visit:
 * https://jestjs.io/docs/configuration
 */

import type { Config } from 'jest';
import nextJest from 'next/jest.js';

const createJestConfig = nextJest({
  // Provide the path to your Next.js app to load next.config.js and .env files in your test environment
  dir: './',
});

// Add any custom config to be passed to Jest
const config: Config = {
  // Indicates whether the coverage information should be collected while executing the test
  collectCoverage: true,

  // The directory where Jest should output its coverage files
  coverageDirectory: 'coverage',

  // Indicates which provider should be used to instrument code for coverage
  coverageProvider: 'v8',

  // The files that should be reviewed for the coverage report
  // The paths starting with ! are ignored because there's not anything to test
  collectCoverageFrom: [
    'src/**/*.{js,jsx,ts,tsx}',
    '!src/data/**',
    '!src/types/**',
    '!src/utils/**',
    '!**/node_modules/**',
  ],

  // The test environment that will be used for testing
  testEnvironment: 'jsdom',

  // Allows the usage of @/ to import files for mocks instead of having to write the whole path
  moduleNameMapper: {
    // Handle module aliases
    '^@/(.*)$': '<rootDir>/src/$1',
  },
};

// Constant createJestConfig is exported this way to ensure that next/jest can load the Next.js config which is async
export default createJestConfig(config);

Feels like the execution order of the test affects it since I've had runs where that test is executed first and it passes. But shouldn't Jest pick up my file that covers this entirely?

I would expect the tests to pass, since they are really covered. Is there an option to make Jest execute only towards the specific test files for each file tested, since I cannot find it? I tried debugging with seeds, but it didn't work. I tried running them in random order as well, but it feels like the order gets cached until you make a change.

Again, everything works as it should on Windows, but when ran on Mac or Ubuntu CI it behaves strangely. I tried reproducing it on Windows, setting up the project from scratch, same package and package locks.

My guess is that another test is mocking it or something like that, but when I run the Jest clear all mocks and reset functions in the test file, it still persists.

Has anyone had this issue? I couldn't find any threads on this that had replies, only one similar one that wasn't specifically related to my case and had no answers.


Solution

  • After analyzing the situation and testing around, it turns out that for some reason the following row in the Jest settings was problematic:

    coverageProvider: 'v8',
    

    There is some kind of issue going on with that provider and it is not picking up the code coverage appropriately.

    The solution was to change to babel:

    coverageProvider: 'babel',
    

    Which gave a 100% across all environments (as it should). I did not notice any negative differences in execution time, I'd even say that it seems faster compared to the previous results.

    I will look into opening an issue with v8.