Test Reports and Dashboards for the Whole Team

1. Why the CI log alone isn't enough

In most teams, the CI pipeline is the only source of test results: a green or red icon next to the commit, a long console log that gets overwritten by the next run. For the developer who triggered the job, that is often enough. For everyone else on the team, product owners, QA leads, new colleagues, that log is practically invisible: it sits behind a login, is unstructured, and doesn't answer any of the questions that actually matter. How many tests have become flakier over the last two weeks? Is the suite getting slower? Which tests fail repeatedly without anyone noticing?

The core problem isn't a lack of information, it's a lack of presentation. A CI log answers whether the current run is green or red, but not how healthy the test suite is overall. Without separate reports, notifications, and dashboards, test quality becomes something that only gets looked at when it fails, reactive instead of proactive. That exact gap is closed by HTML reports, chat notifications, and trend dashboards, which we build up step by step below.

2. Generating HTML reports: Cypress mochawesome and Playwright

The first and most important step is a searchable HTML report instead of plain console text. For Cypress, mochawesome has become the de facto standard: the reporter produces a JSON file per test run with all results plus screenshots on failure, which is then merged into a single file with mochawesome-merge and turned into a navigable HTML page with mochawesome-report-generator. Playwright ships an HTML reporter out of the box: playwright show-report opens an interactive local report with traces, screenshots, and videos for every failed test, with no extra dependencies required.

The key is to persist the report as a CI artifact, not just generate it locally. In GitHub Actions, GitLab CI, or Jenkins, the generated HTML directory can be uploaded as an artifact and reached via a link in the pipeline result, so every team member can open the full report including traces without reproducing the pipeline locally. Running a JSON reporter alongside also lays the data foundation for the trend dashboards covered in section 4.

// playwright.config.js: enable both HTML and JSON reporters
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  retries: 2,
  reporter: [
    // Human-readable report for local and CI artifact viewing
    ['html', { outputFolder: 'playwright-report', open: 'never' }],
    // Machine-readable results for trend dashboards and flakiness tracking
    ['json', { outputFile: 'test-results/results.json' }],
    ['github'],
  ],
  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

3. Slack and Teams notifications for failures and daily summaries

HTML reports solve the depth-of-detail problem, but nobody opens them proactively unless someone points them there. A Slack or Teams notification straight from the CI pipeline closes that gap: on a failed run against the main branch, a CI step automatically posts a message with the number of failed tests, the affected test names, and a direct link to the HTML report. That way the team learns about a problem within seconds, instead of discovering it the next time someone happens to check the pipeline manually.

Just as important as failure notifications is a daily summary, independent of status. A scheduled CI job, for example once every morning, posts pass rate, number of flaky tests, and total duration of the last overnight runs into a dedicated channel. This routine message fundamentally changes how the team perceives test quality: it becomes a visible, daily-present figure instead of something that only comes into focus after a failure.

# GitHub Actions step: post a Slack summary after the E2E test job
- name: Notify Slack on test result
  if: always()
  uses: slackapi/slack-github-action@v1.27.0
  with:
    payload: |
      {
        "text": "E2E tests ${{ job.status }}: ${{ env.PASS_COUNT }} passed, ${{ env.FAIL_COUNT }} failed, ${{ env.FLAKY_COUNT }} flaky. Report: ${{ env.REPORT_URL }}"
      }
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

4. Trend dashboards: pass rate and duration over time

A single report shows the state of one run, not how things develop over time. Tools like Allure Report or Currents.dev aggregate historical test results into a dashboard with time series for pass rate, average duration per test, and the trend of overall suite duration. Allure generates trend charts over the last N runs automatically from the same JSON result files used for the HTML reports, including categorization of failure causes. Currents is built specifically for Cypress and Playwright and additionally provides duration breakdowns per CI runner, which matters for parallelized suites.

Teams that don't want to introduce a third-party tool can build a minimal dashboard of their own: every CI run writes a compact JSON object with timestamp, pass rate, duration, and flake count into a data directory or a small database, and a simple script aggregates that history into a trend dataset rendered as a chart on an internal page. The effort is modest, but the effect is significant: creeping regressions become visible long before they turn into acute outages.

{
  "reporter": "mochawesome",
  "reporterOptions": {
    "reportDir": "cypress/reports/mocha",
    "overwrite": false,
    "html": false,
    "json": true,
    "quiet": true
  }
}

5. Tracking flakiness rate as a first-class metric

Pass and fail counts alone obscure a central problem: a test that fails on the first attempt but turns green after an automatic retry shows up in a simple statistic as "passed", even though it's unreliable. Cypress and Playwright both support automatic retries, which masks failures in the CI interface without actually fixing the underlying problem. A dedicated flakiness rate, defined as the share of tests that only turned green after at least one retry, makes that hidden risk visible and measurable.

In practice, a simple tagging system pays off: any test that repeatedly shows up as flaky gets labeled and tracked in a dedicated list reviewed weekly. Playwright logs retry attempts by default in its JSON reporter via the retry field, so the flakiness rate can be calculated directly from existing report data without extra instrumentation. The goal isn't zero flakiness at any cost, but a metric that keeps trending down instead of quietly growing.

6. Catching duration trends before the suite gets too slow

Test suites rarely get slow all at once, but rather through many small slowdowns: a new test with unnecessarily long waits, an extra API call in a setup hook, a selector waiting on a sluggish element. Each individual change barely stands out in the CI log, but over months a suite's duration can double or triple without anyone noticing, until the pipeline noticeably becomes a drag on the entire development process.

A duration trend per test, not just per suite, surfaces such regressions early. Playwright and Cypress both log the duration of every individual test in the JSON report, so outliers and creeping slowdowns can be extracted with little effort. A sensible CI gate mechanism automatically warns when the suite's total duration exceeds a defined threshold relative to the rolling average of recent runs, instead of discovering the slowdown months later in a retrospective.

// scripts/aggregate-test-trends.js
// Reads historical Playwright JSON reports and builds a trend dataset
import { readdirSync, readFileSync, writeFileSync } from 'node:fs';
import path from 'node:path';

const resultsDir = 'test-results/history';
const trend = readdirSync(resultsDir)
  .filter((file) => file.endsWith('.json'))
  .map((file) => {
    const data = JSON.parse(readFileSync(path.join(resultsDir, file), 'utf-8'));
    const total = data.suites.flatMap((s) => s.specs).length;
    const flaky = data.suites.flatMap((s) => s.specs)
      .filter((spec) => spec.tests.some((t) => t.results.length > 1)).length;

    return {
      date: data.stats.startTime,
      durationMs: data.stats.duration,
      passRate: data.stats.expected / total,
      flakinessRate: flaky / total,
    };
  })
  .sort((a, b) => new Date(a.date) - new Date(b.date));

writeFileSync('dashboard/trend.json', JSON.stringify(trend, null, 2));

7. Making test health visible: dashboard instead of manual checking

Even a good trend dashboard achieves little if it merely exists and nobody actively opens it. The most effective lever is treating test health with the same visibility as production monitoring: a dashboard on a screen in the office, or a permanently linked landing page in the internal wiki, showing pass rate, flakiness rate, and duration trend at all times. That constant presence noticeably changes team behavior: a slightly declining trend gets noticed immediately, instead of only surfacing once the suite is completely unstable.

For remote teams, the same principle works through a pinned message in the team chat or an automatically updated wiki page rewritten once a day by a CI job. What matters less is the specific tool, and more the rule behind it: test health is a team metric like uptime or error rate in production, not a detail known only to the developer who happened to see the last red run.

8. Integrating test reports into pull request comments and checks

The most effective moment for test feedback is when a developer still has their code in context, which is directly inside the pull request. A CI step that automatically posts a comment with a summary, the number of failed tests, and a link to the full HTML report into the pull request skips the detour through the separate CI interface and makes test results a fixed part of the code review process.

On top of that, GitHub Checks or GitLab merge request widgets can annotate individual test results directly on the affected code line, for example via Playwright's official GitHub Actions reporter. That lets a reviewer instantly see which test failed and why, without switching context. This tight integration significantly lowers the barrier to taking test results seriously, because they appear in the same window as the code diff itself.

# GitHub Actions: upload the HTML report and comment the summary on the PR
- name: Upload Playwright HTML report
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: playwright-report
    path: playwright-report/
    retention-days: 14

- name: Comment test summary on pull request
  if: always() && github.event_name == 'pull_request'
  uses: marocchino/sticky-pull-request-comment@v2
  with:
    message: |
      ### E2E Test Report
      Pass rate: ${{ env.PASS_RATE }}%
      Flaky tests: ${{ env.FLAKY_COUNT }}
      [View full HTML report](${{ env.REPORT_ARTIFACT_URL }})

9. Reporting methods compared

The reporting layers covered here, HTML report, chat notification, trend dashboard, flakiness metric, and PR integration, complement each other rather than replacing one another. The table below shows the weakness of each individual method and which setup addresses it.

In practice, a robust setup combines all five layers: HTML reports as the detail source, chat notifications as an instant alert, a trend dashboard as a long-range radar, a flakiness rate as an early-warning system, and PR integration so nobody has to search for results manually. Running just one of these layers only ever covers a fraction of the visibility a team actually needs.

10. Summary

Test reports and dashboards for the whole team solve a structural problem: a green CI checkmark says nothing about the actual health of a test suite. HTML reports with mochawesome or the Playwright reporter provide the depth of detail, Slack and Teams notifications provide instant visibility on failures and in daily summaries, and trend dashboards with tools like Allure or Currents make development over weeks and months visible instead of just the last run.

The decisive cultural shift, though, lies not in the tooling but in the routine behind it: flakiness rate and duration trend become metrics that are discussed regularly by the team, much like uptime or error rates in production. Consistently integrating test results into pull requests and keeping them on a shared, visible dashboard prevents test quality from only becoming a topic during a crisis.

11. FAQ: Test Reports and Dashboards for the Team