Same working Playwright tests fail when placed in a project in playwright.config.js

I have 75 or so tests of my React app that all pass just fine with the following config:

export default defineConfig({
  testDir: 'test',
  outputDir: 'test-results',
  fullyParallel: true,
  testMatch: '**/SecurityPrechecks.test.js',
  use: {
    baseURL: 'http://localhost:5173',
  },
  webServer: {
    command: 'npx vite --host 0.0.0.0 --mode test',
    port: 5173,
    reuseExistingServer: !process.env.CI,
  },
  reporter: [
    ['list'],
  ],
});

The tests themselves use the goto('/') method to navigate and this is run with playwright test.

I need to put this in a project so that I can us a different webServer setup for some new tests. But before I even get to that, the exact same configuration in a project makes each test fail immediately. Here's the config:

export default defineConfig({
  testDir: 'test',
  outputDir: 'test-results',
  fullyParallel: true,
  projects: [
    {
      name: 'security-wrapper',
      testMatch: '**/SecurityPrechecks.test.js',
      use: {
        baseURL: 'http://localhost:5173',
      },
      webServer: {
        command: 'npx vite --host 0.0.0.0 --mode test',
        port: 5173,
        reuseExistingServer: !process.env.CI,
      },
    }
  ],
  reporter: [
    ['list'],
  ],
});

Run with playwright test --project=security-wrapper

The errors with the project setup are:

Error: page.goto: net::ERR_CONNECTION_REFUSED at http://localhost:5173/
    Call log:
      - navigating to "http://localhost:5173/", waiting until "load"

What I've tried:

1. All kids of timing approaches of waiting before page.goto('/').
2. Changing --host to 127.0.0.1
3. Running concurrently with the command command: 'npx concurrently -k -s first "npx vite --host 127.0.0.1 --mode test" "npx wait-on http://127.0.0.1:5173"',
4. running --debug just gives the same error as the terminal output, always fails immediately on page.goto('/')
5. running linear, not parallel

Every time, they fail just as fast with the same error message. Ideas?

Turns out that webServer needs to be in the top level context for defineConfig (source). That's why it was never starting up when included in the projects array and failing immediately. You can launch multiple webservers, but they can't be isolated per-project with the projects array. This is a requested feature.

My specific case involved one project that tested against a live QA server, and the other against the local dev server. The linked github issue shows a nice workaround to use ENV vars to change the value of webServer as needed, but I opted to just let the dev server run every time, regardless of test, so that running all tests with playwright test would always work. The live QA server was defined in those tests specifically. My final configuration was:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: 'test',
  outputDir: 'test-results',
  fullyParallel: true,
  webServer: {
      command: 'npx vite --host 0.0.0.0 --mode test',
      port: 5173,
      reuseExistingServer: !process.env.CI,
    },
  projects: [
    {
      name: 'security-headers',
      testMatch: '**/security-headers.test.js',
      use: {
        baseURL: undefined,
      },
    },
    {
      name: 'security-wrapper',
      testMatch: '**/SecurityPrechecks.test.js',
      use: {
        baseURL: 'http://localhost:5173',
      },
    }
  ],
  reporter: [
    ['list'],
  ],
});

and is called in my package.json with:

    "test:headers": "playwright test --project=security-headers",
    "test:wrapper": "playwright test --project=security-wrapper",
    "test:all": "playwright test",