visual-studiologgingnunitnunittestadapter

Get Nunit Test Order after Execution to analyze brittle Tests

This is not a question about setup and teardown. This is not a question about order attributes.

When I run my Nunit Tests one of them kills a dependency that other tests rely on. My summary shows a lot of failing tests.

A good place to start would be the last passing test working my way downward through subsequent failures. I need to look at the log files of the tests or something better.

I am running the tests using the testadapter in visual studio.

Here are my packages:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Castle.Core" version="3.3.3" targetFramework="net452" />
  <package id="NUnit" version="3.5.0" targetFramework="net452" />
  <package id="NUnit3TestAdapter" version="3.6.0" targetFramework="net452" />
  <package id="RhinoMocks" version="3.6.1" targetFramework="net452" />
  <package id="TestStack.White" version="0.13.3" targetFramework="net452" />
</packages>

I can not figure out where the log files of the last testrun are. Searching the web or stack overflow did not help me. I possibly used incorrect search terms.

My problem is that I need to find the one test that is brittle and fails subsequent tests so I can figure out how to fix that.

EDIT: I found somehting about a tracelogger. I can activate it by using the OneTimeSetUp of my very base SetUpFixture. InternalTrace.Initialize(TestContext.CurrentContext.WorkDirectory + "nunittrace.log", InternalTraceLevel.Debug); Sadly that had no real effect the file ony contains one line confirming the initialisation.

EDIT: I do not want to rely on the order of tests or control the order of tests. Marking tests with an order attribute is completely out of the question. What I want is to look at the test order after the execution so i can find what exactly makes these tests brittle in order to fix it by controlling that brittleness. Setup / Teardown, another Interface? all options that I will have once I find the problem.

Solution

  • With a problem of that nature, I would use the console runner rather than the VS adapter. The --labels:Before option will give you a list of all the tests in the order they are executed.

    If you are using any parallel execution, you should obviously turn it off for these test runs. You can do that in the console runner without editing your test code by using --workers=0 on the command-line.