Cross-Browser Testing with Playwright

1. Why cross-browser testing is essential for Magento stores

In most CI pipelines, the E2E suite runs against Chromium by default because it is the fastest and least error-prone setup. For a Magento or Hyvä store that is a dangerous blind spot: checkout, the single most critical piece of code in the whole store, has to work in every browser real customers actually use, not just the one that is most convenient in the pipeline. A single broken checkout flow in Safari costs real revenue, even if the same test was green in Chrome.

The assumption that "if it works in Chrome, it works everywhere" rarely holds up against modern CSS features such as gap in flexbox, :has() selectors, or native form controls. Hyvä stores built with Tailwind CSS and Alpine.js in particular lean on modern browser APIs whose support differs between engines. Playwright solves the structural problem that used to plague Selenium Grid setups with separate drivers per browser: one API, one test codebase, three engines.

2. Chromium, Firefox, and WebKit: how the engines really differ

Behind the three browsers Playwright drives sit three technically independent rendering engines: Blink (Chromium, the base for Chrome, Edge, and most other Chromium browsers), Gecko (Firefox, a fully independent engine developed by Mozilla), and WebKit (Safari, historically related to Blink but developed separately since the 2013 fork). Each engine ships its own JavaScript engine too: V8 for Chromium, SpiderMonkey for Firefox, and JavaScriptCore for WebKit, with measurable differences in the timing of promises, microtasks, and requestAnimationFrame.

What matters for testing in practice: Playwright installs its own patched builds of Chromium, Firefox, and WebKit via npx playwright install, not the Safari app already installed on the system. The WebKit binary is a very close approximation of Safari behavior, but not a perfect match, especially for platform-specific features like Passkeys or certain media codecs. For most rendering and JavaScript bugs, the approximation is close enough to catch genuine cross-browser issues early.

3. Browser-specific quirks that cause real bugs

Certain quirks come up again and again and are worth dedicated test cases: native form controls such as <input type="date"> or <input type="color"> render differently in every browser, and WebKit often cannot drive the native UI in headless mode. Sticky positioning inside scroll containers, the interaction between overflow: hidden and border-radius, and the visibility thresholds of IntersectionObserver for lazy loading all differ measurably between engines.

Especially relevant for Hyvä stores: Alpine.js transitions (x-transition) sometimes fire on a different timing in Firefox than in Chromium, which makes tests using fixed waitForTimeout values flaky. focus-visible outlines for keyboard navigation also differ, which affects accessibility tests. Instead of guessing at every possible quirk, it pays to check real engine-specific bug trackers before writing test cases for them.

import { test, expect } from '@playwright/test';

test('date picker accepts native input format', async ({ page, browserName }) => {
  // WebKit renders <input type="date"> without a native calendar widget in headless mode
  test.skip(browserName === 'webkit', 'Native date picker UI is not testable in headless WebKit');

  await page.goto('/checkout/shipping');
  await page.getByLabel('Delivery date').fill('2026-08-01');
  await expect(page.getByLabel('Delivery date')).toHaveValue('2026-08-01');
});

test('checkout total updates after quantity change', async ({ page, browserName }) => {
  await page.goto('/checkout/cart');
  await page.getByLabel('Quantity').fill('3');
  await page.getByRole('button', { name: 'Update cart' }).click();

  // Firefox fires the change event slightly later than Chromium/WebKit
  const timeout = browserName === 'firefox' ? 8000 : 5000;
  await expect(page.getByTestId('cart-total')).toContainText('149.70', { timeout });
});

import { test, expect } from '@playwright/test';

test('product grid uses flexbox gap consistently across engines', async ({ page, browserName }) => {
  await page.goto('/catalog/sneakers.html');

  const grid = page.getByTestId('product-grid');
  const cards = grid.getByTestId('product-card');
  const first = await cards.nth(0).boundingBox();
  const second = await cards.nth(1).boundingBox();

  if (!first || !second) {
    throw new Error('Product cards not visible');
  }

  // Older WebKit builds (below 14.1) ignored `gap` on flex containers entirely;
  // this guards against a regression that reintroduces a flex-based grid.
  const horizontalGap = second.x - (first.x + first.width);
  expect(horizontalGap).toBeGreaterThanOrEqual(16);

  test.info().annotations.push({
    type: `engine-${browserName}`,
    description: `measured gap: ${horizontalGap}px`,
  });
});

4. Configuring the browser matrix in playwright.config.ts

Playwright expresses the browser matrix through the projects array in playwright.config.ts. Each project pairs a browser type (chromium, firefox, webkit) with optional device presets from devices, for viewport size, user agent, or touch support. testMatch and testIgnore additionally control which test files run per project, useful for running a mobile checkout smoke test on a single mobile Safari project instead of tripling the entire suite.

Local development usually needs only a single project: npx playwright test --project=chromium finishes in seconds and gives fast feedback. The full matrix only kicks in during the CI pipeline, controlled through environment variables or separate CI jobs per browser. That split between fast local feedback and full CI coverage is the single most important lever for keeping cross-browser testing practical without slowing down day-to-day development.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 4 : undefined,
  reporter: [['html', { open: 'never' }], ['github']],

  use: {
    baseURL: process.env.BASE_URL ?? 'https://shop.mironsoft.test',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },

  // Browser matrix: only test what real users actually use
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
    {
      name: 'mobile-safari',
      use: { ...devices['iPhone 14'] },
      // Only run the checkout smoke suite on mobile Safari, not the full suite
      testMatch: /checkout\.smoke\.spec\.ts/,
    },
  ],
});

5. Running tests in parallel across multiple browsers

Playwright parallelizes on two levels by default: through workers, which run test files concurrently, and through projects, which run the same suite against multiple browsers. With fullyParallel: true, even individual tests inside a single file run in parallel rather than sequentially. That also means a three-browser matrix effectively triples runtime unless enough worker capacity is available, a factor that is often underestimated when planning CI costs.

Parallelism also surfaces bugs that stay hidden in serial test runs: shared test data, race conditions when creating test accounts, or API rate limits. For Magento stores, it is worth giving every worker an isolated test customer and, where possible, an isolated database fixture, so parallel browser runs do not overwrite each other's orders or carts. Playwright's --shard flag additionally splits the suite horizontally across multiple CI runners, independent of the browser dimension.

6. Deciding browser coverage from real user data

"We'll just test every browser" sounds thorough, but it is rarely the right strategy. A more realistic approach is to pull the actual browser distribution of your users from Google Analytics 4, Search Console, or server logs and align the test matrix with it. A typical German B2C store sees roughly 60 percent Chrome, 15 to 20 percent Safari (mostly mobile), a few percent Edge, and often under 5 percent Firefox, numbers that can shift significantly depending on the audience.

A pragmatic rule: engines with a traffic share above a defined threshold, say 5 percent, get a dedicated Playwright project in the CI matrix. Everything below that is checked through occasional manual spot checks instead of full automated coverage. That decision should not be made once and forgotten; revisit it quarterly against current analytics data, since browser market share shifts noticeably, especially through mobile trends.

// Pull browser share from a GA4/Search Console export and derive a coverage threshold
const browserShare = [
  { browser: 'Chrome', engine: 'chromium', share: 0.61 },
  { browser: 'Safari', engine: 'webkit', share: 0.19 },
  { browser: 'Edge', engine: 'chromium', share: 0.09 },
  { browser: 'Firefox', engine: 'gecko', share: 0.06 },
  { browser: 'Samsung Internet', engine: 'chromium', share: 0.03 },
  { browser: 'Opera', engine: 'chromium', share: 0.02 },
];

// Only engines above 5% combined traffic get a dedicated Playwright project
const COVERAGE_THRESHOLD = 0.05;

const enginesToTest = browserShare
  .reduce((engines, entry) => {
    const existing = engines.get(entry.engine) ?? 0;
    engines.set(entry.engine, existing + entry.share);
    return engines;
  }, new Map())
  .entries();

for (const [engine, share] of enginesToTest) {
  if (share >= COVERAGE_THRESHOLD) {
    console.log(`Include ${engine} in CI matrix (${(share * 100).toFixed(1)}% of traffic)`);
  } else {
    console.log(`Skip ${engine} in CI, cover via manual smoke test only (${(share * 100).toFixed(1)}%)`);
  }
}

7. CI matrix strategy: weighing coverage against cost

Every extra browser in the matrix multiplies CI minutes and, with them, pipeline cost directly. A proven strategy: run only the fast Chromium suite on every pull request for quick feedback, while the full browser matrix runs overnight or before a release. fail-fast: false in the matrix strategy also ensures that a failing Firefox job does not cancel the still-running WebKit or Chromium jobs, important for seeing all browser-specific failures in a single run instead of chasing them one at a time.

Additional sharding via --shard spreads a suite that is already split by browser further across multiple runners, which reduces wall-clock time but not the total number of runner minutes. Caching browser binaries between pipeline runs and skipping the matrix for documentation-only changes through path filters are the most effective levers for keeping the cost of a real cross-browser matrix under control.

name: E2E Cross-Browser Tests

on:
  pull_request:
    branches: [main]

jobs:
  e2e:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        browser: [chromium, firefox, webkit]
        shard: [1/2, 2/2]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}

      - name: Run Playwright tests
        run: >
          npx playwright test
          --project=${{ matrix.browser }}
          --shard=${{ matrix.shard }}

      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report-${{ matrix.browser }}-${{ strategy.job-index }}
          path: playwright-report/
          retention-days: 14

8. Debugging browser-specific test failures

When a test fails in only one browser, the trace viewer is the fastest way to the root cause: npx playwright show-trace trace.zip opens a timeline with DOM snapshots, network requests, and console logs for exactly the failed run. Locally, the same failure can be reproduced with npx playwright test --headed --project=webkit --debug, which opens the Playwright Inspector and allows step-by-step execution with page.pause().

Most browser-specific failures turn out to be race conditions that one engine resolves faster or slower than another, for example because WebKit prioritizes a layout reflow differently than Chromium. Fixed waitForTimeout calls only mask such issues instead of fixing them; explicit web-first assertions like expect(locator).toBeVisible() are more robust because they retry automatically up to a timeout. Console logs and stack traces also differ between engines in wording and order, which matters when comparing error messages across browsers.

9. Chromium, Firefox, and WebKit compared

Each engine has its own strengths, its own pitfalls, and a different priority in the test matrix. The table below summarizes what actually matters for each browser and how the coverage decision typically plays out.

In practice, the full matrix does not pay off equally for every test suite: visual and interaction tests around checkout benefit the most from real cross-browser coverage, while pure API or backend tests are browser-independent and should not be duplicated across Playwright projects at all.

10. Summary

Cross-browser testing with Playwright addresses a concrete risk: a store tested only in Chromium can still be broken in Safari or Firefox without the CI pipeline ever noticing. Playwright makes that coverage practical because a single test suite runs against Chromium, Firefox, and WebKit through the projects array in playwright.config.ts, without separate drivers or test code per browser. Targeted tests for known quirks such as native form controls, flexbox gap, or transition timing catch exactly the bugs a single-browser suite systematically misses.

The decisive lever for sustainability, though, is not the completeness of the matrix but its justification: browser coverage should be grounded in real analytics data rather than the assumption that every browser matters equally. A fast Chromium suite on every pull request, combined with a full nightly matrix, balances test depth against CI cost, and turns cross-browser testing into a lasting part of the pipeline instead of a one-off chore.

11. FAQ: Cross-Browser Testing with Playwright