selenium-webdriveryamlazure-pipelinesplaywrightmstest

How do I write Visual Studio testlog during Azure Pipeline and save as a artifact


So we have numerous pipelines that run Selenium or Playwright tests overnight.

The test suites are built using C# with selenium or Playwright and MsTest.

The end result does not display the actual steps taken and I understand this is not possible, but is it possible to create a report artifact after the test run with detail as shown in the Visual Studio Test Log?

Below is an example of what we would like to show in the artifact for each test.

TestTheDeliveryFilterOptions
Source: WIP.feature line 5
Duration: 27.9 sec

  Standard Output: 
-> Loading plugin C:\Users\smith-billybob\source\repos\FruitPi\FruitPiRegression\bin\Debug\net8.0\Reqnroll.MSTest.ReqnrollPlugin.dll
-> Loading plugin C:\Users\smith-billybob\source\repos\FruitPi\FruitPiRegression\bin\Debug\net8.0\FruitPiRegression.dll
-> Using default config
Running from Azure
Day 2 so running on Edge


Debug Trace:
GetTheCurrentMethod() for testy.autotester@TestCompany.co.uk
Load page https://www.fruitpi.co.uk/
In procedure LoginATestUser for user testy.autotester@TestCompany.co.uk 
Clicking element By.CssSelector: [test-id='loginUser.button']
Clicking element By.CssSelector: [test-id='loginUser.button']
Wait for element By.CssSelector: [test-id='sidebar.dashboard'] to be visible
In procedure AddCookies
In procedure Visit
Loading https://www.fruitpi.co.uk/home/#/finance
In procedure FilterByDelStatus
Clicking element By.CssSelector: [test-id='finance.showAllFilters']
Clicking element By.CssSelector: [test-id='finance.DelStatus.Dropdown']
Wait for element By.CssSelector: [test-id='dropdown.item'] to be visible
In procedure AllSuppliersAreDel
Wait for element By.CssSelector: [test-id='finance.invoiceList.actionsButton.openInvoice'] to be visible
Pass: condition is true for Assert all suppliers are active


TestContext Messages:
Given I have logged into FruitPi with group user "testy"
-> done: LoginSteps.GivenIHaveLoggedIntoFruitPiWithGroupUser("testy") (8.0s)
And we visit finance page
-> done: FinanceSteps.GivenWeVisitFinancePage() (0.5s)
When we filter by Del status '2'
-> done: FinanceSteps.WhenWeFilterByDelStatus(2) (12.4s)
Then all suppliers are Del
-> done: FinanceSteps.ThenAllSuppliersAreDel() (3.1s)

This log is created when running the test suite locally from Visual Studio. Some of the lines are written by VS and others by using

Trace.WriteLine(theMessage);

Within the code and each test can be viewed via the test log

enter image description here

So what I am asking here is how can I get this sort of output with a YAML pipeline?


Solution

  • Great news, I think we have this figured out.

    After a pipeline run in Azure navigate to Test Plans -> Runs

    enter image description here

    Then select the run you're looking for

    enter image description here

    Double Click on the run and you get the Run Summary page, now double the attachment

    enter image description here

    This can be opened in Visual Studio etc

    enter image description here

    And double clicking each test will show the steps etc in all their glory

    enter image description here

    Nice..