typescriptplaywrightplaywright-testplaywright-typescript

Playwright test based off array running too many times


Absolute novice to Playwright with minimal async experience.

Running the below code ends up creating multiple screen shots (usually between 8-9.)

I understand this is due to the tests running in parallel. Attempts at getting it to run in serial have been unsuccessful. Only solution I've seen is to run this test with the --workers=1 flag.

How might this code be written so as to only run once for each item in the browserTypes array without forcing the workers flag or adjusting the playwright.config file?

import { test, chromium, firefox, webkit } from '@playwright/test';

test("Browser support test", async () => {
    const browserTypes = [chromium, firefox, webkit];
    const now = new Date();
    const timestamp = now.toISOString().replace(/[:.]/g, "-");

    for (const browserType of browserTypes) {
        console.log(`Running: ${browserType.name()}`);

        const browser = await browserType.launch();
        const page = await browser.newPage();

        await page.goto("https://www.whatsmybrowser.org/");
        await page.screenshot({
            path: `tests-screenshots/pw-${browserType.name()}-${timestamp}.png`,
        });

        await browser.close();
    }
});

Image output based on loop

Log of loop activity


UPDATE based on ggorlen's answer

For completeness and future page views. My original config where the issue could be seen:

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },

    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    }
  ]
});

Reducing the config to the bare minimum did fix the issue.

export default defineConfig({
  testDir: './tests',
  reporter: 'html',
  retries: 0
});

Solution

  • Go ahead and adjust the config--don't manage the browser in your tests. Use the page fixture and let Playwright handle the browser setup and teardown for you.

    Similarly, the browser type is defined in your config or on the command line. There is no need to explicitly to const browserTypes = [chromium, firefox, webkit];, just write your tests in a browser-agnostic manner, and run them with the appropriate config.

    From Run tests on different browsers in the docs (as of 1.49.1):

    Playwright can run your tests in multiple browsers and configurations by setting up projects in the config. You can also add different options for each project.

    import { defineConfig, devices } from '@playwright/test';
    
    export default defineConfig({
      projects: [
        /* Test against desktop browsers */
        {
          name: 'chromium',
          use: { ...devices['Desktop Chrome'] },
        },
        {
          name: 'firefox',
          use: { ...devices['Desktop Firefox'] },
        },
        {
          name: 'webkit',
          use: { ...devices['Desktop Safari'] },
        },
        /* Test against mobile viewports. */
        {
          name: 'Mobile Chrome',
          use: { ...devices['Pixel 5'] },
        },
        {
          name: 'Mobile Safari',
          use: { ...devices['iPhone 12'] },
        },
        /* Test against branded browsers. */
        {
          name: 'Google Chrome',
          use: { ...devices['Desktop Chrome'], channel: 'chrome' }, // or 'chrome-beta'
        },
        {
          name: 'Microsoft Edge',
          use: { ...devices['Desktop Edge'], channel: 'msedge' }, // or 'msedge-dev'
        },
      ],
    });
    

    Playwright will run all projects by default.

    $ npx playwright test
    
    Running 7 tests using 5 workers
    
      ✓ [chromium] › example.spec.ts:3:1 › basic test (2s)
      ✓ [firefox] › example.spec.ts:3:1 › basic test (2s)
      ✓ [webkit] › example.spec.ts:3:1 › basic test (2s)
      ✓ [Mobile Chrome] › example.spec.ts:3:1 › basic test (2s)
      ✓ [Mobile Safari] › example.spec.ts:3:1 › basic test (2s)
      ✓ [Google Chrome] › example.spec.ts:3:1 › basic test (2s)
      ✓ [Microsoft Edge] › example.spec.ts:3:1 › basic test (2s)
    

    Use the --project command line option to run a single project.

    $ npx playwright test --project=firefox
    
    Running 1 test using 1 worker
    
      ✓ [firefox] › example.spec.ts:3:1 › basic test (2s)
    

    With this setup, your test would be simply:

    import {test} from "@playwright/test"; // ^1.46.1
    
    test("Browser support test", async ({page, browserName, isMobile}) => {
      const timestamp = new Date().toISOString().replace(/[:.]/g, "-");
      const mobile = isMobile ? "mobile" : "desktop";
      await page.goto("https://www.whatsmybrowser.org/");
      await page.screenshot({
        path: `tests-screenshots/pw-${browserName}-${mobile}-${timestamp}.png`
      });
      // Note: there should be an assertion in a test
    });
    

    With a sample run (I didn't bother installing Edge, so the failure is expected):

    $ npx playwright test a.test.js
    
    Running 7 tests using 4 workers
    
      ✓  1 [Mobile Chrome] › a.test.js:3:1 › Browser support test (2.2s)
      ✓  2 [chromium] › a.test.js:3:1 › Browser support test (2.1s)
      ✓  3 [webkit] › a.test.js:3:1 › Browser support test (2.6s)
      ✓  4 [firefox] › a.test.js:3:1 › Browser support test (1.9s)
      ✓  5 [Mobile Safari] › a.test.js:3:1 › Browser support test (1.7s)
      ✓  6 [Google Chrome] › a.test.js:3:1 › Browser support test (1.4s)
      ✘  7 [Microsoft Edge] › a.test.js:3:1 › Browser support test (5ms)
    
    
      1) [Microsoft Edge] › a.test.js:3:1 › Browser support test ───────────────────
    
        Error: browserType.launch: Chromium distribution 'msedge' is not found at /opt/microsoft/msedge/msedge
        Run "npx playwright install msedge"
    
      1 failed
        [Microsoft Edge] › a.test.js:3:1 › Browser support test ────────────────────
      6 passed (5.0s)
    

    or with one browser:

     npx playwright test a.test.js --project=firefox
    
    Running 1 test using 1 worker
    
      ✓  1 [firefox] › a.test.js:3:1 › Browser support test (1.7s)
    
      1 passed (2.9s)
    

    If you want to define a special test that would be the only one run on different browsers, you can use tags and/or skip tests based on browser type.


    As for the particular code in your question, I can't reproduce your results, so you might need to share your config (or just don't bother debugging it and use the above approach):

    $ npx playwright test a.test.js # exact code copied from the question
    
    Running 1 test using 1 worker
    
      ✓  1 a.test.js:3:1 › Browser support test (4.9s)
    Running: chromium
    Running: firefox
    Running: webkit
    
      1 passed (5.3s)
    playwright-tests$ ls tests-screenshots/
    pw-chromium-2025-01-09T06-47-23-740Z.png
    pw-firefox-2025-01-09T06-47-23-740Z.png
    pw-webkit-2025-01-09T06-47-23-740Z.png
    

    I don't think the worker count will make a difference--only one worker can possibly run this one test. Multiple workers only come into effect when there are multiple tests and files.