Ensuring Test Isolation Between Test Runs

1. Why tests pass individually but fail in the suite

Tests that pass in isolation, say with npx playwright test login.spec.ts, but fail when the full suite runs almost always point to shared state. The test runner executes the same environment, the same browser context, or the same database for multiple tests back to back, and an earlier test leaves traces that a later test unknowingly depends on or is disrupted by. Classic symptoms: a login test fails because a previous test already set a session cookie, or a checkout test can no longer find a product because another test deleted it beforehand.

The instinct to simply add cy.wait() or increase retries when these failures show up only treats the symptom, not the cause. Isolation means every test establishes its own starting state, regardless of which tests ran before it and in what order. This is not a minor detail, it is the fundamental prerequisite for parallel execution, reliable CI pipelines, and tests that actually surface a bug in the product code instead of a bug in the test setup.

2. Isolating browser storage: localStorage, sessionStorage, cookies

Playwright and Cypress both start every test with an empty state by default, but as soon as tests explicitly reuse page.context() or cy.session() is configured carelessly, localStorage entries, sessionStorage, and cookies survive between tests. A cart token left in localStorage by a previous test causes a subsequent test to suddenly find a populated cart instead of an empty one, even though the test code never expects that.

Playwright isolates storage cleanly through its own BrowserContext instance per test, Cypress requires explicit cy.clearCookies(), cy.clearLocalStorage(), and since version 12 cy.session() with a correctly set cacheAcrossSpecs option. It is also important to keep an eye on IndexedDB and service worker caches, since standard cleanup commands often miss them and stale cached responses can otherwise mask real bugs.

// Playwright: isolated BrowserContext per test, storage is never shared
import { test, expect } from '@playwright/test';

test.describe('Cart', () => {
  test('starts with an empty cart', async ({ browser }) => {
    // A new context creates fresh storage, cookies, and cache
    const context = await browser.newContext();
    const page = await context.newPage();

    await page.goto('/checkout/cart');
    const cartItemCount = await page.locator('[data-testid="cart-item"]').count();
    expect(cartItemCount).toBe(0);

    await context.close();
  });
});

// Cypress: explicitly reset storage before every test
beforeEach(() => {
  cy.clearCookies();
  cy.clearLocalStorage();
  cy.window().then((win) => win.sessionStorage.clear());
});

3. Isolating database and backend state per test

Browser-side isolation is not enough when a test runs against a real Magento instance backed by a MySQL database. An E2E test that places an order changes real database rows, and without teardown, hundreds of test runs accumulate orphaned customers, orders, and products that skew later tests, for example when a test checks the number of orders a customer has placed.

Proven strategies include database transactions that get rolled back after every test, dedicated seed scripts that establish a defined starting state before every run, and fixtures created through the API rather than directly in the database, to stay closer to real user behavior. For Magento projects, a command that loads a database snapshot before the suite runs works well, combined with targeted cleanup of individual entities after each test instead of a full reset, which takes far too long against large catalogs.

#!/usr/bin/env bash
# Before the suite: load a defined database snapshot
mysql -u magento -p"$DB_PASSWORD" magento < fixtures/snapshot-baseline.sql

# Per test: open a transaction, set a savepoint
mysql -u magento -p"$DB_PASSWORD" magento -e "START TRANSACTION; SAVEPOINT test_start;"

# After the test: roll back to the savepoint instead of a full reset
mysql -u magento -p"$DB_PASSWORD" magento -e "ROLLBACK TO SAVEPOINT test_start;"

# Alternative: delete individual entities created during the test
bin/magento customer:delete --email="e2e-*@mironsoft.test"

4. Writing order-independent tests

A reliable sign of order dependency: a test fails as soon as it runs with --shard or in a randomized order, even though it always passed in the original file order. Both Playwright and Cypress let you deliberately run tests in random order, which reliably surfaces such hidden dependencies long before they become a problem in production CI.

The rule is easy to state but easy to violate: no test may rely on the side effects of another test, whether on data it created or on a particular application state. In practice, that means giving every test its own beforeEach setup that establishes all required preconditions itself, instead of relying on the incidental order of the test file. Test suites that are regularly and deliberately randomized force the team to build in isolation from the start instead of retrofitting it later.

5. Parallel-safe test data: unique values per worker

Parallel test workers often share the same backend instance, which is why fixed test data like test@example.com or the SKU TEST-001 is guaranteed to cause collisions as soon as two workers create the same customer or product at the same time. This usually shows up as a sporadic 409 conflict error or as an incorrect record that another worker overwrote, making it a classic flaky test candidate.

The fix is a data factory that generates unique values for every test run, typically from a combination of worker index, timestamp, and a random component. Playwright exposes testInfo.workerIndex directly, and Cypress offers a similar mechanism via Cypress.env() with a parallel ID from the CI runner. The key is to use these unique values consistently for every test-critical field: email addresses, SKUs, customer numbers, and order references, so that two parallel workers never claim the same record.

// Playwright: generate unique test data per worker
import { test as base } from '@playwright/test';

type TestData = { email: string; sku: string; customerNo: string };

export const test = base.extend<{ testData: TestData }>({
  testData: async ({}, use, testInfo) => {
    const uniqueSuffix = `${testInfo.workerIndex}-${Date.now()}`;

    const data: TestData = {
      email: `e2e-${uniqueSuffix}@mironsoft.test`,
      sku: `TEST-SKU-${uniqueSuffix}`,
      customerNo: `CUST-${uniqueSuffix}`,
    };

    await use(data);
  },
});

test('creates a customer with a unique email', async ({ page, testData }) => {
  await page.goto('/customer/account/create');
  await page.fill('#email', testData.email);
  // ... remaining test flow
});

6. Cypress- and Playwright-specific isolation mechanisms

Playwright isolates by default through the BrowserContext, a lightweight, fully separate browser session with its own storage, cookies, and cache, recreated for every test or every test file. This is the decisive structural difference from older versions of Cypress, which by default reused a single browser tab across an entire spec file.

Since version 12, Cypress has retrofitted a comparable concept with cy.session(): a login flow runs once and the resulting state, cookies, localStorage, and sessionStorage, gets cached under a cache key and validated when needed, instead of logging in through the UI again for every test. Without explicit cy.session() or targeted cy.clearCookies() in beforeEach, Cypress tests within the same spec file share the same browser state, which effectively defeats isolation and produces exactly the symptoms from section one.

// Cypress: cy.session for fast, isolated login state
Cypress.Commands.add('loginAsCustomer', (email, password) => {
  cy.session(
    [email, password],
    () => {
      cy.visit('/customer/account/login');
      cy.get('#email').type(email);
      cy.get('#pass').type(password);
      cy.get('#send2').click();
      cy.url().should('include', '/customer/account');
    },
    {
      // Validate before every reuse that the session is still valid
      validate: () => {
        cy.getCookie('PHPSESSID').should('exist');
      },
      cacheAcrossSpecs: false,
    }
  );
});

it('shows the customer area after login', () => {
  cy.loginAsCustomer('e2e-worker0@mironsoft.test', 'Test1234!');
  cy.visit('/customer/account');
  cy.contains('My Account').should('be.visible');
});

7. Teardown and cleanup strategies for test data

Cleanup after a test matters just as much as setup before it, but it is neglected more often, because a missing teardown step rarely breaks the current test run visibly, it just contaminates later runs. A robust pattern is to track every record created through the API with a unique ID and delete it explicitly in an afterEach or afterAll hook, instead of relying on a global database reset that costs minutes on large test suites.

Playwright fixtures with automatic teardown via the use() function and Cypress tasks that talk directly to the backend via cy.task() are both suitable places for cleanup logic, since they run regardless of the test outcome, even if an assertion fails. A nightly CI job that identifies and cleans up orphaned test data by a name prefix like e2e- adds a safety net for cases where an aborted test run skipped the regular teardown.

# CI pipeline: parallel shards plus a cleanup job as a safety net
jobs:
  e2e-tests:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - run: npx playwright test --shard=${{ matrix.shard }}/4
    env:
      CI: true

  nightly-cleanup:
    needs: e2e-tests
    if: always()
    steps:
      - name: Remove orphaned test data
        run: bin/magento customer:delete --email-prefix="e2e-" --older-than="24h"

8. Diagnosing flaky tests: shared state as the root cause

A test that fails roughly five times out of a hundred runs, with no change to the test code or the application, is the most reliable indicator of shared state. The first diagnostic step is to run the suspect test in isolation ten times in a row, then run the same test ten times in parallel alongside other tests that touch the same resource, such as customer data or the product catalog, to check whether the failure rate rises with concurrency.

Playwright traces and Cypress videos from failed CI runs often show the decisive clue: an unexpected state at the start of the test, such as an already logged-in user or a cart that should have been empty. A logging statement that records the current state of cookies, localStorage, and relevant database rows at the start of every test surfaces shared-state leaks within a few CI runs, instead of days of spot-checking.

9. Isolation patterns compared: bad vs. good

The table below sets common anti-patterns in test isolation against the recommended alternatives, and shows why seemingly convenient shortcuts lead to unstable suites over time.

In practice these patterns reinforce each other: if test data isn't unique, tests can't safely run in parallel, and without parallelization you lose the speed advantage that makes Cypress and Playwright attractive in CI pipelines in the first place. Consistent isolation across all four layers, storage, database, test data, and ordering, is therefore not a nice-to-have, it is the prerequisite for fast, reliable E2E suites.

10. Summary

Test isolation between test runs solves a core problem: unstable suites cost trust in the test system and CI time through unnecessary retries. Isolating browser storage through dedicated contexts or clean cy.session() usage prevents login or cart state from leaking between tests. Isolating database state through transaction rollback instead of full resets keeps test runs fast while still staying clean.

Parallel-safe test data with unique emails, SKUs, and customer numbers per worker is the prerequisite for parallelization to actually deliver a speed advantage instead of creating new collisions. Consistent cleanup after every test and a nightly safety-net job prevent orphaned test data from accumulating over weeks and silently skewing later runs.

11. FAQ: Test Isolation Between Test Runs