So we have numerous pipelines that run Selenium or Playwright tests overnight.
The test suites are built using C# with selenium or Playwright and MsTest.
The end result does not display the actual steps taken and I understand this is not possible, but is it possible to create a report artifact after the test run with detail as shown in the Visual Studio Test Log?
Below is an example of what we would like to show in the artifact for each test.
TestTheDeliveryFilterOptions
Source: WIP.feature line 5
Duration: 27.9 sec
Standard Output:
-> Loading plugin C:\Users\smith-billybob\source\repos\FruitPi\FruitPiRegression\bin\Debug\net8.0\Reqnroll.MSTest.ReqnrollPlugin.dll
-> Loading plugin C:\Users\smith-billybob\source\repos\FruitPi\FruitPiRegression\bin\Debug\net8.0\FruitPiRegression.dll
-> Using default config
Running from Azure
Day 2 so running on Edge
Debug Trace:
GetTheCurrentMethod() for testy.autotester@TestCompany.co.uk
Load page https://www.fruitpi.co.uk/
In procedure LoginATestUser for user testy.autotester@TestCompany.co.uk
Clicking element By.CssSelector: [test-id='loginUser.button']
Clicking element By.CssSelector: [test-id='loginUser.button']
Wait for element By.CssSelector: [test-id='sidebar.dashboard'] to be visible
In procedure AddCookies
In procedure Visit
Loading https://www.fruitpi.co.uk/home/#/finance
In procedure FilterByDelStatus
Clicking element By.CssSelector: [test-id='finance.showAllFilters']
Clicking element By.CssSelector: [test-id='finance.DelStatus.Dropdown']
Wait for element By.CssSelector: [test-id='dropdown.item'] to be visible
In procedure AllSuppliersAreDel
Wait for element By.CssSelector: [test-id='finance.invoiceList.actionsButton.openInvoice'] to be visible
Pass: condition is true for Assert all suppliers are active
TestContext Messages:
Given I have logged into FruitPi with group user "testy"
-> done: LoginSteps.GivenIHaveLoggedIntoFruitPiWithGroupUser("testy") (8.0s)
And we visit finance page
-> done: FinanceSteps.GivenWeVisitFinancePage() (0.5s)
When we filter by Del status '2'
-> done: FinanceSteps.WhenWeFilterByDelStatus(2) (12.4s)
Then all suppliers are Del
-> done: FinanceSteps.ThenAllSuppliersAreDel() (3.1s)
This log is created when running the test suite locally from Visual Studio. Some of the lines are written by VS and others by using
Trace.WriteLine(theMessage);
Within the code and each test can be viewed via the test log
So what I am asking here is how can I get this sort of output with a YAML pipeline?
Great news, I think we have this figured out.
After a pipeline run in Azure navigate to Test Plans -> Runs
Then select the run you're looking for
Double Click on the run and you get the Run Summary page, now double the attachment
This can be opened in Visual Studio etc
And double clicking each test will show the steps etc in all their glory
Nice..