Establishing Visual Regression Testing in the Hyvä Theme

1. Why visual regression testing needs its own layer next to functional tests

Functional end-to-end tests check whether an interaction produces the expected result, for example whether clicking Add to Cart actually makes an item show up in the mini cart. What they fundamentally don't check is what that interaction looks like, which means a Tailwind refactor that accidentally changes a utility class like px-4 to p-4 and shifts a layout goes unnoticed by the functional suite as long as the button remains clickable.

Visual regression testing closes exactly that gap by comparing a reference screenshot of a page or component against a new screenshot taken after a code change, pixel by pixel or perceptually, and flagging any significant deviation. For a Tailwind-heavy theme like Hyvä, where CSS changes can easily ripple out to unrelated parts of the markup without anyone noticing, this extra layer isn't a luxury, it's a necessary safety net.

2. Choosing a tool: weighing Playwright screenshots against Percy

Playwright's built-in toHaveScreenshot() assertion compares images pixel by pixel against a baseline stored in the repository and needs no external service, which means it runs at no extra cost and stays fully under a team's own control. The downside is that baselines can render slightly differently depending on the operating system and font rendering, so screenshots need to be generated consistently inside the same Docker image the CI pipeline uses.

Percy, on the other hand, is a cloud service that runs a perceptual rather than a purely pixel-based comparison, making it less sensitive to tiny rendering differences, but it comes with ongoing cost and a dependency on an external vendor. For a single, manageable theme project the pixel-based Playwright approach is often enough, while Percy pays off more for larger teams with an established review workflow for visual diffs.

// playwright.config.ts
export default defineConfig({
  expect: {
    toHaveScreenshot: {
      maxDiffPixelRatio: 0.01,
      animations: 'disabled',
    },
  },
  projects: [
    { name: 'visual-chromium', use: { browserName: 'chromium' } },
  ],
});

3. Choosing a sensible set of pages for the test scope

The biggest mistake when starting out with visual regression testing is capturing every single product page in a catalog with thousands of items, even though most product pages share the same template and therefore the same potential source of bugs. A small, deliberately curated set of templates works far better: a product detail page with and without a special price, a category page with an open facet panel, a cart with several items, a single checkout step, and the CMS homepage.

That set covers every layout pattern present in the theme at least once, without overloading the pipeline with hundreds of nearly identical screenshots that would fail together anyway whenever a shared component like the product card changes. New page types, such as a newly introduced landing page layout, only get added once they genuinely represent a distinct pattern, not for every new URL.

4. Reliably masking dynamic content like prices and dates

A price that changes between two test runs because of an automatic discount rule, or a relative date like three days ago on a product review, causes a screenshot comparison to be flagged as a regression even though the layout itself never changed. Playwright's mask option on toHaveScreenshot() lets you cover exactly those areas with a solid block through a selector, instead of making the entire screenshot unreliable.

Percy follows the same idea through percy-css rules that hide certain elements or fill them with a fixed replacement color before the screenshot gets captured. In both cases the actual layout area where the price or date sits still stays part of the comparison, so a real layout bug there still gets caught, while the constantly changing content itself no longer triggers false alarms.

await expect(page).toHaveScreenshot('pdp-backpack-classic.png', {
  mask: [
    page.locator('[data-testid="product-price"]'),
    page.locator('[data-testid="review-date"]'),
  ],
  maxDiffPixelRatio: 0.01,
});

5. Deterministic test data as the foundation of stable snapshots

Masking alone doesn't fix everything, because a price that switches from a two-digit to a three-digit number between two test runs can produce a slightly different text width despite the mask, nudging neighboring elements out of place. A dedicated fixture product with a fixed price that no discount rule ever touches avoids this problem at the root instead of papering over it with masks afterward.

For the catalog, it pays off to keep a small, clearly marked set of test products that gets recreated reproducibly through its own CLI command before every visual test run, instead of relying on whatever production data happens to exist or on manually maintained staging content that can quietly drift over time.

6. Web fonts and lazy loading as a hidden source of flakiness

A screenshot taken before a web font has fully loaded briefly shows the operating system's fallback font with a different character width, which shifts line breaks into unexpected places and fails the comparison even though nothing is actually broken. An await page.evaluate(() => document.fonts.ready) right before every screenshot guarantees that the final font has genuinely rendered.

Lazy loading of product images causes a similar issue, since an image outside the initially visible viewport still shows as a gray placeholder at the moment the screenshot is taken. A short scroll across the entire page before the actual capture, together with disabling CSS transitions through the animations option, keeps the timing of the capture itself from becoming a source of errors.

7. Establishing an approval workflow for genuine visual changes

Not every detected deviation is a bug. A deliberate design change, such as more spacing between product cards after a redesign, rightfully produces a visual diff that should be accepted rather than fixed. Playwright handles this through the --update-snapshots command, which promotes the new capture to the new baseline and lands it in the same commit as the code change, instead of as a separate, easily forgotten step.

Percy offers its own web interface for this, where a reviewer explicitly marks each diff as approved or rejected before the associated merge request is allowed to merge. Regardless of which tool is used, this decision should always be made by a person who understands the business context of the change, never automatically or by the pipeline itself.

8. Integrating visual tests into CI without slowing everything down

Because visual tests are naturally slower than pure functional tests, since a full screenshot has to be rendered and compared for every page under test, a separate, parallel pipeline job usually pays off better than folding it into the already-running functional suite. That job can additionally be scoped to only trigger on merge requests that actually touch .phtml or CSS files, instead of running against the whole repository on every single commit.

For very large themes, a sharding strategy that splits the curated page list across several parallel runners also pays off, keeping total runtime manageable even as the number of snapshot pages grows, so visual regression testing never becomes the limiting factor for the whole pipeline.

9. Practical example: locking down the category page with facet navigation

A category page with an open facet panel is a particularly worthwhile target for visual regression testing, because it combines several distinct Hyvä components at once: filter checkboxes, a price slider, a sort dropdown and the product card grid. A single screenshot therefore covers potential layout bugs in several components simultaneously, instead of needing an isolated test for each one.

It matters to capture the screenshot only after the facet panel has fully opened and its Alpine transition has completed, so the image reflects the same final state a user would actually see, instead of freezing an arbitrary intermediate frame somewhere in the middle of the animation.

test('category page with open facets stays visually stable', async ({ page }) => {
  await page.goto('/women/jackets.html');
  await page.evaluate(() => (window as any).__alpineReady);
  await page.locator('[data-testid="filter-toggle"]').click();
  await page.locator('[data-testid="filter-panel"]').waitFor({ state: 'visible' });
  await page.evaluate(() => document.fonts.ready);

  await expect(page).toHaveScreenshot('plp-jackets-facets-open.png', {
    mask: [page.locator('[data-testid="product-price"]')],
    fullPage: true,
  });
});

10. Summary

11. FAQ: Visual Regression Testing in the Hyvä Theme: Key Takeaways