Accessibility Monitoring in Production With Lighthouse CI

1. Continuous monitoring instead of a one-off audit

A classic accessibility audit is usually commissioned once, or at long intervals, and produces a detailed, often manually written report that marks the state of a page on exactly one day. In practice, months routinely pass between two such audits, during which numerous new features, component redesigns, and third-party integrations go live, each one capable of reintroducing a previously fixed barrier without anyone noticing.

Monitoring takes a different approach. Instead of checking at a single point in time, the accessibility state gets measured automatically on every single deployment and plotted as a trend line over time, so a decline shows up right after the deployment that caused it, not at the next big audit months later. Lighthouse CI fits this well because it wires directly into existing deployment pipelines and needs no separate, manually triggered process.

2. Lighthouse CI versus Lighthouse in DevTools

The Lighthouse tool familiar from Chrome DevTools runs a single, manually triggered check in the browser and shows the result right in the developer interface, with each result standing alone and never automatically compared to a previous run. Lighthouse CI, by contrast, is a standalone command line tool that runs the same Lighthouse engine repeatedly and automatically, stores the results historically, and checks them against previously defined thresholds.

That historical record is the key difference. A single DevTools run only produces a snapshot, while Lighthouse CI turns repeated runs into a continuous time series that real trends and sudden outliers can be read from. Lighthouse CI also runs each check multiple times by default and uses the median, to smooth out measurement noise from system load or network conditions.

3. Integration into the deployment pipeline

In a GitLab CI pipeline for a Magento and Hyvä project, Lighthouse CI can be wired in as its own job after the build and before the actual deployment to a staging environment, with the job automatically failing as soon as the measured accessibility score drops below the defined threshold. That does not just surface a regression, it actively blocks it from ever reaching production.

It matters to run the job against an actually running instance populated with real data, such as a staging environment with a realistic product catalog, rather than against an empty development page, because many accessibility-relevant issues only surface with real content: long product names, populated shopping carts, or actual images.

# .gitlab-ci.yml
lighthouse-a11y:
  stage: test
  image: node:20
  script:
    - npm install -g @lhci/cli
    - lhci autorun --config=./lighthouserc.js
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  allow_failure: false

4. Defining thresholds and budgets for the accessibility score

The lighthouserc.js configuration file defines through assertions what minimum accessibility score counts as passing, and a realistic starting value should be based on the page's actually measured current state instead of immediately forcing a theoretical ideal of a hundred points that a grown store can barely reach. A sensible approach is to set the threshold slightly below the current actual state at first and raise it step by step as concrete improvements land.

Beyond the global score, Lighthouse CI also lets individual audits get marked as mandatory criteria, such as sufficient color contrast or present form labels, so a team can deliberately decide which specific issues must never reach production, even while the overall score formally still clears the threshold.

// lighthouserc.js
module.exports = {
  ci: {
    collect: {
      url: ['https://staging.example.test/product/sample-item.html'],
      numberOfRuns: 3,
    },
    assert: {
      assertions: {
        'categories:accessibility': ['error', { minScore: 0.92 }],
        'color-contrast': ['error', { minScore: 1 }],
        'label': ['error', { minScore: 1 }],
      },
    },
    upload: { target: 'temporary-public-storage' },
  },
};

5. Regression detection on new features

The real value of Lighthouse CI shows up less in the absolute score than in comparing two consecutive measurements. If a merge request's accessibility score drops noticeably against the target branch's state, that points to a concrete regression, even if the absolute value still clears the general threshold. This relative check catches issues a plain threshold comparison would miss entirely.

Rolling out new components, such as a new product filter or a reworked checkout step, is a good moment for a targeted comparison of the affected page before and after the change, because new interactive elements especially tend to introduce missing labels, insufficient contrast, or controls that cannot be focused, without any of that being obvious during everyday manual testing.

6. What the Lighthouse accessibility score does not cover

Much like axe-core, the Lighthouse accessibility score is based on a limited set of criteria that can be checked automatically, and it therefore covers only part of the WCAG success criteria. Keyboard operability across the whole page, a logical focus order, or the actual quality of error messages cannot be captured by an automated score, which is why a high Lighthouse value must never be treated as equal to full accessibility.

For that reason, Lighthouse CI never replaces a manual audit or testing with real users. It complements both by adding continuous, automated observation that holds a baseline quality between the bigger manual reviews and reliably surfaces sudden technical regressions.

7. Establishing reporting and alerting across the team

Lighthouse CI can upload results to a dedicated Lighthouse CI server or a compatible storage service, building a searchable history of every run that can be charted across weeks and months. That visibility matters, because it keeps accessibility from fading into an abstract background goal and instead turns it into a concrete, observable metric sitting right next to performance metrics like Largest Contentful Paint on the same dashboard.

It is also worth setting up a notification, for example through a Slack or Teams webhook, that fires as soon as a merge request drops below the defined threshold, so the responsible team can react immediately instead of only discovering the regression at the next scheduled reporting meeting.

8. Combining Lighthouse CI and an axe-core CI runner

Lighthouse CI and a dedicated axe-core CI runner do not compete, they check different layers. Lighthouse delivers an aggregated score across multiple categories and is particularly well suited for watching a trend over time, while an axe-core runner delivers detailed, individual findings with an exact element and rule and is better suited for targeted bug fixing.

In a mature pipeline, both tools run in parallel. Lighthouse CI watches the long-term trend and blocks on a significant score drop, while the axe-core runner delivers the concrete list of violations on every run that a developer can act on directly. That combination pairs macro-level monitoring with precise micro-level diagnosis.

9. Practical example: continuous monitoring of a product detail page

For a Magento product detail page in the Hyvä theme, Lighthouse CI can be configured so that every merge request automatically triggers three runs against a representative sample URL and the median accessibility score gets compared against the main branch's last known value. If the score drops by more than two percentage points, the job fails and the merge request stays blocked until the cause is identified.

In practice, a setup like this has repeatedly shown that a new review component or an extra cross-selling carousel introduced insufficient contrast or missing focus indicators, long before those components ever reached production, meaning the fix could happen while the feature was still being built rather than after launch.

10. Summary

11. FAQ: Lighthouse CI Monitoring: Key Takeaways