haskellcabal

Cabal test coverage report is empty

Here is my project structure:

.git
src/
  exe/
    Main.hs
  lib/
    ModuleA/
    ModuleB/
    UnitTests/
      UnitTests.hs
    MainLib.hs
giter.cabal

I have an executable, and the only thing it does is call the main function form the library. Here is the contents of src/exe/Main.hs:

import qualified MainLib

main :: IO ()
main = MainLib.main

Here are the relevant parts of the giter.cabal file:

library 
    import: shared-properties
    exposed-modules: MainLib
    other-modules:
        ModuleA
        ModuleB

    build-depends:
      , brick
      , directory
      , fsnotify

    hs-source-dirs:   src/lib

executable giter
    import: shared-properties
    main-is:          Main.hs
    other-modules:

    build-depends:
        , giter
        , base
        , brick
    hs-source-dirs:   src/exe

test-suite giter-test
    type:             exitcode-stdio-1.0
    hs-source-dirs:   src/lib
    main-is:          UnitTests/UnitTests.hs
    default-language: Haskell2010
    other-modules:
        ModuleA
        ModuleB
    build-depends:
        , base
        , directory
        , HUnit
        , QuickCheck

Here is the result of cabal new-test all --enable-coverage:

.
.
.
[2 of 2] Linking /home/refaelsh/repos/giter/dist-newstyle/build/x86_64-linux/ghc-9.4.6/giter-0.1.0/build/giter/giter [Library changed]
Running 1 test suites...
Test suite giter-test: RUNNING...
Tests
  Events.hs
    handleSpecialKey_CommitOpen:   OK
    handleSpecialKey_CommitClosed: OK
    booleanDecision_True:          OK
    booleanDecision_False:         OK
  MainList.hs
    mainList_getUnstagedSlices:    OK
      +++ OK, passed 100 tests.
    mainList_getStagedSlices:      OK
      +++ OK, passed 100 tests.
    mainList_getTreeSlices:        OK (0.78s)
      +++ OK, passed 100 tests.

All 7 tests passed (0.81s)
Test suite giter-test: PASS
Test suite logged to:
/home/refaelsh/repos/giter/dist-newstyle/build/x86_64-linux/ghc-9.4.6/giter-0.1.0/test/giter-0.1.0-giter-test.log
Writing: hpc_index.html
Writing: hpc_index_fun.html
Writing: hpc_index_alt.html
Writing: hpc_index_exp.html
Test coverage report written to
/home/refaelsh/repos/giter/dist-newstyle/build/x86_64-linux/ghc-9.4.6/giter-0.1.0/hpc/vanilla/html/giter-test/hpc_index.html
1 of 1 test suites (1 of 1 test cases) passed.
Writing: hpc_index.html
Writing: hpc_index_fun.html
Writing: hpc_index_alt.html
Writing: hpc_index_exp.html
Package coverage report written to
/home/refaelsh/repos/giter/dist-newstyle/build/x86_64-linux/ghc-9.4.6/giter-0.1.0/hpc/vanilla/html/giter-0.1.0/hpc_index.html

Questions:

  1. Both HTML files are empty, here is one of them (they both look the same):
module  Top Level Definitions   Alternatives    Expressions
%   covered / total %   covered / total %   covered / total
  Program Coverage Total    -   0/0     -   0/0     -   0/0

Why they are empty?

  1. Why there are 2 HTMLs files? I was expecting one, only for the library part of the project.
Solution

  • Both HTML files are empty, here is one of them

    ...

    Why they are empty?

    The shorter answer:

    Modules in the test suite's other-modules list are excluded from the reports.

    The simple solution:

    In your repository, just remove all of the other-modules from your test suite. Here is what that looks like:

    test-suite giter-test
    type:             exitcode-stdio-1.0
    hs-source-dirs:   src/lib
    main-is:          UnitTests/UnitTests.hs
    default-language: Haskell2010

    build-depends:
        , base
        , directory
        , HUnit
        , QuickCheck
        , split
        , tasty
        , tasty-hunit
        , tasty-quickcheck

    The longer answer:

    In your unit tests, when you import ModuleA or ModuleB in the Main module defined in src/lib/UnitTests/UnitTests.hs, those modules are imported in a way that causes them to NOT be tracked by Haskell Program Coverage (the tool that is enabled by the --enable-coverage flag).

    When you run cabal new-test --enable-coverage, ModuleA is not compiled with support for Haskell Program Coverage. This is because ModuleA is one of the other-modules. In order to compile ModuleA with support for Haskell Program Coverage, you must remove ModuleA from the other-modules list of your test suite.

    That allows you to compile ModuleA with support for Haskell Program Coverage.

    Now, you should see some content for ModuleA in the generated reports. You can do the same for ModuleB.

    I hope this helps :)

    Why there are 2 HTMLs files? I was expecting one, only for the library part of the project.

    The .../giter-test/hpc_index.html file is a report of the test coverage of the test-suite giter-test. It reports how much of the code is tested by the test-suite giter-test.

    On the other hand, the .../giter-0.1.0/hpc_index.html file is a report of the test coverage of ALL the test suites. If you add another test suite that covers different parts of your code, then you will see a 3rd report for that new test suite, and the .../giter-0.1.0/hpc_index.html file would contain an aggregation of the reports for all of the test suites.