Currently I have Playwright tests that I am running inside Docker container both locally and in the CI. Unfortunately the tests fail on the CI with screenshot comparison error.
Upon investigation I can see that the fonts on the CI screenshots are different from the ones used to generate the expected images (also run inside docker). But this doesn't happen consistently - sometimes it happens, sometimes it doesn't. Using sharding shows that some shards pass fine and others don't inside the same job run.
Figured it out - after reviewing the trace file - it turned out that the fonts are loaded from our CDN which is on a different domain from the local one inside the Docker container where the tests are running. This results in the following CORS error and the fonts are not applied.
Access to font at 'https://mycdn.com/fonts/my-font.woff2' from origin 'http://localhost:8003' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Furthermore this issue was strangely affecting only the tests running on chromium and NOT the ones on webkit.
After some more research I found out this Playwright issue suggesting some parameters when running tests in chromium browser which actually did the job!
So the final solution was to change the chromium configuration to:
{
name: 'chromium',
use: {
browserName: 'chromium',
bypassCSP: true,
launchOptions: {
args: ['--disable-web-security'],
},
},
},