jenkinsjunit

Continue Build on Jenkins when JUnit stage fails


Given: I have a jenkinsfile with multiple stages running different types of unit tests.

// first set of unit tests
stage("konsist (architecture rules)") {
    try {
        fastlaneUtils.run("konsist") // this runs the "gradlew :konsist:test
    } finally {
        dir('../') {
            junit(
                testResults: '**/test-results/**/*.xml'
            )
        }
    }
}
// second set of unit tests
stage("unit tests") {
    try {
        fastlaneUtils.run("test") // runs all other test
    } finally {
        dir('../') {
            junit(
                testResults: '**/test-results/**/*.xml'
            )
        }
    }
}

Goal: I would like to continue my build, when the first unit test stage (konsist stage) fails but still mark the whole build as unstable.


Solution

  • There are two parts to the solution.

    First, run the konsist stage, catch the exception and pass the skipMarkingBuildUnstable flag to the junit() call and mark the build manually as "unstable" (which will let the build continue).

    try {
        fastlaneUtils.run("konsist")
    } catch(e) {
        dir('../') {
            junit(
                testResults: '**/test-results/**/*.xml',
                skipMarkingBuildUnstable: true
            )
            unstable(message: "konsist failed")
        }
    }
    

    Now the problem is, that the unit tests stage will also collect the reports from the konsist stage and also list the failed test cases from there.

    Therefore, the second part of the solution is to set a custom junit-reports directory for the konsist module. Assuming you are using gradle this can be done like this:

    tasks.test {
        // Gradle's default behavior assumes that a module's code is up-to-date if the module itself hasn't been modified.
        // Since konsist needs to rerun whenever any other modules changes we force to always rerun it
        outputs.upToDateWhen { false }
        // put reports in separate folder so jenkins can evaluate them separately
        reports {
            html.required.set(true)
            html.outputLocation.set(layout.buildDirectory.dir("reports/konsist/html"))
    
            junitXml.required.set(true)
            junitXml.outputLocation.set(layout.buildDirectory.dir("konsist-results/xml"))
        }
    }
    

    Then you can adapt the konsist stage like this:

    try {
        fastlaneUtils.run("konsist")
    } catch(e) {
        dir('../') {
            junit(
                testResults: '**/konsist-results/**/*.xml',
                skipMarkingBuildUnstable: true
            )
            unstable(message: "konsist failed")
        }
    }
    

    Note that there are different ways to deal with the second problem (like cleaning up after running the konsist stage). The approach presented here works well, if you can configure the results directory of the JUnit tests.