I have a project with tests split in unit and integration phases. I have it running buildbot and the problem is that even in tests are failing maven return code is 0 so buildbot build is succesful.
This is the result of mvn integration-test:
Results :
Tests in error:
Info about failed tests
Tests run: 5, Failures: 0, Errors: 5, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 19 seconds
[INFO] Finished at: Tue Feb 12 09:43:53 UTC 2013
[INFO] Final Memory: 36M/97M
[INFO] ------------------------------------------------------------------------
$ echo $?
0
Result for mvn install is the same without the build successful part Results :
Tests in error:
Info about failed tests
Tests run: 5, Failures: 0, Errors: 5, Skipped: 0
$ echo $?
0
Surefire configuration is like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<printSummary>true</printSummary>
<excludedGroups>com.testlib.IntegrationTest</excludedGroups>
</configuration>
<executions>
<execution>
<id>unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludedGroups>com.testlib.IntegrationTest</excludedGroups>
</configuration>
</execution>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<groups>com.testlib.IntegrationTest</groups>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I've reading other threads about maven return codes but in theory related bugs should be fixed in my maven version (Apache Maven 2.2.1 (rdebian-8))
Is there any way to change this behaviour?
Update: As suggested I tried with surefire:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.13</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.13</version>
</dependency>
</dependencies>
<configuration>
<groups>com.testlib.IntegrationTest</groups>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/*.class</include>
</includes>
</configuration>
</execution>
</executions>
I need the surefire-junit to avoid initialization errors.
I managed to have it working with two different configurations, using groups and using folders using just surefire
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration><excludedGroups>com.testlib.IntegrationTest</excludedGroups>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludedGroups>com.biicode.testlib.UnitTest</excludedGroups>
<groups>com.testlib.IntegrationTest</groups>
</configuration>
</execution>
</executions>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/integration/*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/integration/*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>