We are using TeamCity for Selenium tests execution with Specflow+Specrun, the problem is that TeamCity count re-executed tests.
For example if some test failed for first time he will be re-executed two times more, in teamcity we will see that three tests failed, but probably it was one test.
Also if first re-execution will fail, but other two will success, this will be reported in teamcity as two failed, one passed, but I need to be reported that only one test has been passed.
It is possible to configure this in TeamCity using service messages or something else?
UPDATED:
Based on answer we can gather logs using powershell script and change build status using teamcity service messages:
$path = Get-ChildItem %teamcity.build.checkoutDir%\ProjectFolder\bin\Remote\TestResults\specrun.log
$file = Get-content $path
$total = $file | select-string "Total:"
$passed = $file | select-string "Succeeded:"
$failed = $file | select-string "Failed:"
write-host $( "##teamcity[buildStatus text='Tests {0}, {1}, {2}']" -f $total, $passed, $failed )
TeamCity is behaving as expected. TeamCity reports what Testing Framework tells him. Roughly the process is:
You have to configure your Testing Framework (specflow+specrun in your case) to change the way it reports the re-execution. You should configure specflow to log the re-executions in a different way... if it can.
In general you should avoid re-execution of tests. The tests should work at the first attempt. They should prepare the state if needed, do their test, and clean the state.
The need of re-execution indicates probably some dependency with the other tests. Something like:
The correct way is that TEST001 does not have a dependency of TEST002, adds the user Foo1, test search and clean user Foo1.