I want to run the build for Apache James which has a huge test suite that is running very long due to tests that irrelevant to me, e.g. tests concerning RabbitMQ. Thus I'd like to exclude those and I want to do so from the command line (not by editing POMs). I'm using Maven 3.6.3 on Java 11 OpenJDK. The project uses JUnit5 and maven-surefire-plugin 2.22.2.
Now, I would expect the following to work:
For example, to run only test methods in the org.example.MyTest test class you can execute mvn -Dtest=org.example.MyTest test from the command line.
But it doesn't work. In fact, as soon as I set the test parameter to anything else than an empty string, all tests will be skipped. I tried some of the syntax that is supposedly supported...
mvn package -Dtest=*
mvn package -Dtest=".*"
mvn package -Dtest=\!SomethingFishy
mvn package -Dtest='!MavenHeadache'
mvn package -Dtest='!%regex[.*HelpMe.*]'
...but the result is always the same:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test
(default-test) on project testing-base: No tests were executed!
(Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]
I'm running the package goal, as it does the full build, but test behaves the same. Also tried specifying -Dincludes=... / -Dexcludes=..., which had no effect at all: All tests are executed regardless and the parameters don't even show up in the output of mvn -X .... This behavior doesn't change when update maven-surefire-plugin to the latest version which is 3.0.0-M5.
Do I understand something wrong here? How can i specify inclusions/exclusions in this setup?
Update: It looks like this is caused by nested projects and/or James' project structure in particular. If I enter a "leaf project", e.g. core, then inclusions/exclusions begin to work:
cd core
mvn test -Dtest=HostTest # will only run HostTest, as expected
mvn test -Dtest=\!HostTest # will run all tests but HostTest, as expected
As suggested by RobertScholte, I have looked at the maven-surefire-plugin configuration, but couldn't find anything that seems to be related to this behavior.
The error message tells what you need to know: in project testing-base which is the first to run, there's no test matching your pattern, so it fails to ensure you won't have a false impression of success.
It then suggests to use -DfailIfNoTests=false option to let maven ignore modules that don't have any test matching the pattern (it's probably what you need).