I am attempting to calculate the Test Strength generated by the HTML output from the data within the XML output of PIT however the figure I calculate doesn't align with the Test Strength from the HTML report.
I use the equation (mutationsKilled / (totalMutations - mutationsNotCovered) * 100)
, where:
mutationsKilled
was generated by counting the entries where status = "KILLED"
totalMutations
was generated by counting the total entriesmutationsNotCovered
was generated by counting the entries where detected = "false"
However, it's the last of these that I suspect could be the problem, as when looking back at the raw XML output I find that detected metric may not necessarily indicate whether the mutation was covered or not.
Is there a tried and tested way of doing this? Also may be worth mentioning I'm running PIT 1.9.4 and also the incremental analysis mode, if this has any bearing on the issue.
After experimenting and intentionally reducing my test coverage to produce uncovered mutants, I found that there is a possible status, NO_COVERAGE
. By adjusting the above equation to generate mutationsNotCovered
by counting the entries where status = "NO_COVERAGE"
, the correct statistics are produced.