junitjunit5spock

Group (or merge) @Unroll tests from Spock in JUnit XML result file


I have a Spock @Unroll test like this:

@Unroll
def 'A session uses the TLS protocol #enabledProtocolsOfSocket #protocolOfSession'() {
    given:
    // ...

    expect:
    // ...

    where:
    enabledProtocolsOfSocket | protocolOfSession
    ["TLSv1.3"]              | "TLSv1.3"
    ["TLSv1.2", "TLSv1.3"]   | "TLSv1.3"
    ["TLSv1.2"]              | "TLSv1.2"
    ["TLSv1.2", "TLSv1.1"]   | "TLSv1.2"
    ["TLSv1.1", "TLSv1.2"]   | "TLSv1.2"
    ["TLSv1.1"]              | "NONE"
    ["TLSv1"]                | "NONE"
    ["TLSv1", "TLSv1.1"]     | "NONE"
    ["TLSv1.1", "TLSv1"]     | "NONE"
}

which leads to this JUnit result XML:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="com.acme.TlsProtocolSpec" tests="9" skipped="0" failures="0" errors="0" timestamp="2024-07-19T17:23:47" hostname="acme-build5eb650" time="0.403">
  <properties/>
  <testcase name="A session uses the TLS protocol [TLSv1.3] TLSv1.3" classname="com.acme.TlsProtocolSpec" time="0.081"/>
  <testcase name="A session uses the TLS protocol [TLSv1.2, TLSv1.3] TLSv1.3" classname="com.acme.TlsProtocolSpec" time="0.037"/>
  <testcase name="A session uses the TLS protocol [TLSv1.2] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.084"/>
  <testcase name="A session uses the TLS protocol [TLSv1.2, TLSv1.1] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.07"/>
  <testcase name="A session uses the TLS protocol [TLSv1.1, TLSv1.2] TLSv1.2" classname="com.acme.TlsProtocolSpec" time="0.071"/>
  <testcase name="A session uses the TLS protocol [TLSv1.1] NONE" classname="com.acme.TlsProtocolSpec" time="0.014"/>
  <testcase name="A session uses the TLS protocol [TLSv1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
  <testcase name="A session uses the TLS protocol [TLSv1, TLSv1.1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
  <testcase name="A session uses the TLS protocol [TLSv1.1, TLSv1] NONE" classname="com.acme.TlsProtocolSpec" time="0.013"/>
  <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

The problem is: I cannot map these results to the test in post-processing the result file – there is no result named A session uses the TLS protocol #enabledProtocolsOfSocket #protocolOfSession (due to the placeholders) and there are no tests named A session uses the TLS protocol [TLSv1.2, TLSv1.3] TLSv1.3 or ... or ... (due to the placeholders being replaced).

Is there some feature to "group" or "merge" these test runs into one result, named as the test? I'm thinking of something like xUnit's <collection> Element, which contains one <test> child node for each test run, like so:

<collection total="9" passed="9" failed="0" skipped="0" name="Test collection for Com.Acme.SomeClass.Tests.SomeTestName" time="0.484">
    <test name="Com.Acme.SomeClass.Tests.SomeTestName(someParam: &quot;10&quot)" ...>
    </test>
    <test name="Com.Acme.SomeClass.Tests.SomeTestName(someParam: &quot;11&quot)" ...>
    </test>
    ...
</collection>

But any solution to preserve the relation of the generically named test to all test runs (i.e. if just one test run fails, the test is reported failing) is welcome, be it by configuring the JUnit output or by some Spock feature.

Update

As @leonard-brünings pointed out: The build tool is of interest here, as it produces said output. I'm currently using Gradle 8.5 (Groovy 3.0.17, Spock 2.4-M4-groovy-4.0), configured like this:

test {
    reports {
        junitXml.required = true
        html.required = true
    }

    useJUnitPlatform()
}

I searched a bit in that direction and found settings like mergeReruns (which sounds helpful but is for failed and then retried runs?) and plugins like The Test Report Aggregation Plugin (which aggregates by type, not by testcase?) here. So at least these do not help.


Solution

  • Seems that @Unroll is obsolete since Spock 2.x:

    A simple @Unroll without argument is not needed anymore [...], so any simple @Unroll annotations can be removed from existing code.

    But/and: There's @Rollup now, which reports a single <testcase> now, which is what I was looking for.

    On the downside: Those rolled up tests do not show up individually in the IDE neither; all those #myParam1-placeholders in the test's name are obsolete now, as well (I guess).