tagsbuild.gradleserenity-bddcucumber-serenity

Serenity BDD, Java and Gradle not enforcing Cucumber Tags

I am attempting to limit my Serenity BDD suite with the use of tags.

Having tried many, many variations (WithTags, CucumberOptions, -Dcucumber.options, -Dcucumber.filter.tags) without joy, turning to the wisdom of the crowd.

Currently I am executing within IntelliJ IDE.
I am using a run configuration for Gradle and passing in the tags.

IntelliJ Run Configuration: IntelliJ screenshot

In previous challenge of setting environment, I found with Gradle, I had to pass the parameter along using the Gradle build file:

test {
    useJUnitPlatform()
    def parameter = '-Denvironment='
    def environment = System.getProperty('environment')
    
    jvmArgs parameter+environment
}

When I passed the cucumber.filter.tags in addition to environment, and it didn't work, I replicated this approach:

test {
    useJUnitPlatform()
    def parameter = '-Denvironment='
    def environment = System.getProperty('environment')
    def parameterCucumber = '-Dcucumber.filter.tags="'
    def tags = System.getProperty('cucumber.filter.tags')

    if (tags=="null" || tags.isEmpty()) {
        println("Calling JVM with following arguments:")
        println(parameter+environment)
        jvmArgs parameter+environment
    } else {
        println("Calling JVM with following arguments:")
        println(parameter+environment+" "+parameterCucumber+tags+"\"")
        jvmArgs parameter+environment+" "+parameterCucumber+tags+"\""
    }
}

When I run the Gradle build in IntelliJ it will output the following due to println() statement. It appears to be the correct JVMArguments:

> Configure project :
Calling JVM with following arguments:
-Denvironment=sandbox -Dcucumber.filter.tags="@sprint-13"
> Task :clearReports
> Task :clean
> Task :compileJava
> Task :processResources
> Task :classes
> Task :compileTestJava
> Task :processTestResources
> Task :testClasses
14:56:54.470 [Test worker] DEBUG i.c.core.plugin.SerenityReporter - SRP:handleTestRunStarted Thread[Test worker,5,main] 
14:56:54.727 [Test worker] DEBUG i.c.core.plugin.SerenityReporter - SRP:handleTestSourceRead Thread[Test worker,5,main]
14:56:54.728 [Test worker] DEBUG i.c.core.plugin.SerenityReporter - Running feature from classpath:features//closingPackage/closingpackage.feature
14:56:54.748 [Test worker] INFO  n.thucydides.core.steps.StepEventBus - Loading JUnit Platform configuration parameters from classpath resource [junit-platform.properties].
14:56:54.790 [Test worker] INFO   -

When I execute with IntelliJ it runs all the tests as can be seen in this snippet of the Serenity Report

Serenity list of executed tags: tags

Expectation: Only scenarios with the @sprint-13 tag would run. Expectation: This is core functionality and I must be missing something everyone else gets ;)

What did I try?

Solution

  • TL;DR Gradle requires some groovy statements to make this work. When working with negating a tag, or using multiple tags, must pass those enclosed in (parentheses)

    Details: When using Serenity BDD, Java, Cucumber, and Gradle, I found that it was necessary to add code to the test Task in build.gradle in order to have both an environment setting and cucumber tags passed to the Java Virtual machine. The code needed to parse command line arguments and then call the method jvmArgs passing a COMMA DELIMITED set of parameters.

    As seen in the post, previously I was passing just a single string containing multiple parameters to jvmArgs. Instead I needed multiple strings that were comma delimited

    The corrected jvmArgs based code

    test {
    useJUnitPlatform()

    jvmArgs "-Denvironment="+System.getProperty('environment'), "-Dcucumber.filter.tags=\"" + System.getProperty('cucumber.filter.tags') + "\""
}

    This will work correctly when the IntelliJ Gradle build has the run syntax

    clean test -Denvironment=sandbox -Dcucumber.filter.tags="@sprint29"

    Which on a command line alone would be

    ./gradlew clean test -Denvironment=sandbox -Dcucumber.filter.tags="@sprint29"

    I then had issues with multiple paramters or negating the tag I was using. While dignosing I found two things

    1. Multiple tags or negation need parenthesis. Such as -Dcucumber.filter.tags="(not @sprint29)" or -Dcucumber.filter.tags="(@sprint22 or @sprint29)"
    2. Rather than using jvmArgs I should use systemProperty

    This left me with the following in build.gradle under the test task

    test {
    useJUnitPlatform()


    def parameter = '-Denvironment='
    def environment = System.getProperty('environment')
    def parameterCucumber = '-Dcucumber.filter.tags="'
    def tags = System.getProperty('cucumber.filter.tags')

    //We don't want to pass the tags unless they are passed in.  Otherwise we say run all the blank tags and nothing
    //  executes
    if (tags == null || tags.isEmpty()) {
        println("No cucumber tags found in run command.  Running all tests.")
        println("Environment tag found: " + parameter + environment)
        println("Setting environment -> systemProperty 'environment', '" + environment + "'")

        //systemProperty, not System.setProperty
        systemProperty 'environment', environment
    } else {
        println("Cucumber tags found: " + parameterCucumber + "'" + tags + "'")
        println("Setting cucumber tags -> systemProperty 'cucumber.filter.tags', " + "'" + tags + "'")
        println("Environment tag found: " + parameter + environment)
        println("Setting environment -> systemProperty 'environment', '" + environment + "'")

        systemProperty 'environment', environment
        systemProperty 'cucumber.filter.tags', tags
    }

    From an IntelliJ Run/Debug Configuration a valid Run command clean test -Denvironment=sandbox -Dcucumber.filter.tags="(not @skipDev)"

    Other valid tag syntax, in IntelliJ, or via command line, look like