Reliably Testing Email Delivery in E2E Tests

1. Why email delivery poses its own challenge in E2E tests

A browser test can directly observe whatever appears in the DOM once an action was performed, but an outgoing email leaves that observable area entirely: it gets handed off from the application to a mail server, queued there, and eventually, with an unknown delay, actually delivered. Without deliberate access to that email, a test is left with only indirect checks, say whether an application log entry about the successful send appears, which says nothing about whether the actual email content was correct, complete, and carried the right links.

This gap causes many projects to skip automated email content testing entirely, checking it only manually and sporadically, even though order confirmations, password reset emails, and registration emails are exactly the messages where a broken link or a wrongly interpolated placeholder has a direct, tangible effect on customers. A test SMTP server closes this gap by intercepting the email within the same test environment, before it ever reaches a real mail server.

2. Test SMTP servers at a glance: Mailhog, Mailpit, and Mailtrap

Mailhog and its actively maintained successor Mailpit are lightweight, self-hosted SMTP servers that intercept every incoming email, never actually deliver it, and instead expose it through a web interface as well as a REST API, making them ideal to run as a container within a local Docker environment or a CI pipeline, without ever talking to a real, public mail server. Mailtrap follows a similar concept but is typically used as a hosted service and additionally offers spam-score analysis and HTML validation for the email content.

For a Magento and Hyva development environment running in Docker, Mailpit is a particularly good fit, since it can run as an additional service within the same compose file, works entirely offline, and is therefore dependent neither on an external internet connection nor on the rate limits of a hosted provider, making CI runs considerably more stable.

3. Switching Magento and Hyva to a test SMTP server

Magento uses PHP's built-in mail function or a configured SMTP transport by default, which means the entire outgoing email traffic in a test environment can be redirected to the local test SMTP server with just a handful of configuration values, without changing anything about the application's actual delivery logic.

# Switch Magento to a local test SMTP server (Mailpit)
bin/magento config:set system/smtp/host mailpit
bin/magento config:set system/smtp/port 1025
bin/magento config:set system/smtp/transport smtp

# As its own service in compose.dev.yaml:
#   mailpit:
#     image: axllent/mailpit
#     ports:
#       - "8025:8025"   # web UI and REST API
#       - "1025:1025"   # SMTP port

4. Fetching emails via the Mailpit API in a test

Instead of having to open a mailbox in the browser, an E2E test queries the test SMTP server's REST API directly, filters the most recently arrived message by recipient address, and then checks subject, HTML content, and contained links against the expected values, exactly like an assertion against a DOM element.

This API access happens entirely outside the browser context, which is why it can be added to Playwright or Cypress without friction as an extra HTTP request within the same test case, right after the triggering action, say completing an order, was performed in the browser.

import { test, expect } from '@playwright/test';

test('order confirmation contains the correct order number and link', async ({ page, request }) => {
  await page.goto('/checkout/onepage/success');
  const orderNumber = await page.locator('[data-testid="order-number"]').textContent();

  const search = await request.get(
    `http://mailpit:8025/api/v1/search?query=to:customer@example.test`
  );
  const { messages } = await search.json();
  const latest = messages[0];

  const detail = await request.get(`http://mailpit:8025/api/v1/message/${latest.ID}`);
  const email = await detail.json();

  expect(email.Subject).toContain('Your Order');
  expect(email.HTML).toContain(orderNumber);
  expect(email.HTML).toMatch(/href="https:\/\/[^"]+\/sales\/order\/view/);
});

5. Deliberately validating content and links in order and registration emails

Beyond a plain delivery check, it pays off to take a targeted look at three especially error-prone parts of every transactional email: correctly interpolated placeholders like customer name and order number, the complete and syntactically valid URL of every contained link, and the presence of every expected line item in an order confirmation with multiple products.

For registration emails carrying a confirmation link, a true end-to-end check is valuable, one where the test actually opens the extracted link in the browser and then verifies the account is marked as activated afterward, instead of relying solely on the link's textual presence in the email content, since an incorrectly signed or already expired token would not show up under a plain text check.

If an email additionally contains a file attachment, say an invoice as a PDF on an order confirmation, the test shouldn't just check that the attachment exists, it should actually extract the attachment from the test SMTP server's response and inspect it with the same parsing technique used for direct file downloads, to make sure the attachment is substantively correct and not, say, an empty or corrupted file.

Since many email clients additionally display a plain-text alternative alongside the HTML version, it's worth adding a check that also verifies the email's text body against the same core information like order number and total amount, since a message only correctly maintained in the HTML version would appear incomplete or broken in text-only email clients.

6. Avoiding flakiness from asynchronous delivery

The most common cause of unstable email tests is a fixed, estimated wait time between triggering the action and fetching the email, say a flat two-second wait that's sometimes enough and sometimes not depending on system load, causing the test to fail for no real reason occasionally in the CI pipeline, even though the email actually arrives shortly after.

The reliable solution is polling with a clearly defined timeout, repeatedly querying the test SMTP server's API at short intervals until either the expected message is found or a generous but finite time limit is exceeded, letting the test continue immediately after the email arrives in the normal case and only exhaust the full wait time in an actual failure case.

async function waitForEmail(request, toAddress, timeoutMs = 10000) {
  const start = Date.now();
  while (Date.now() - start < timeoutMs) {
    const res = await request.get(`http://mailpit:8025/api/v1/search?query=to:${toAddress}`);
    const { messages } = await res.json();
    if (messages.length > 0) return messages[0];
    await new Promise((resolve) => setTimeout(resolve, 300));
  }
  throw new Error(`No email to ${toAddress} arrived within ${timeoutMs}ms`);
}

7. Reliably distinguishing multiple emails within the same test

As soon as a test scenario triggers more than one email, say an order confirmation followed by a separate shipping notification, a simple fetch of the latest message to a given address is no longer sufficient, since both messages go to the same address and their arrival order under test load can't always be reliably predicted.

In this case, the test filters specifically by subject or a unique identifier contained in the email, say the order number, instead of blindly relying on arrival order, letting every email be uniquely identified and deliberately checked regardless of send time or arrival sequence, even when several similar messages get generated in quick succession within the same test run.

8. Running a test SMTP server as a service container in the CI pipeline

In a GitLab CI or GitHub Actions pipeline, the test SMTP server runs as an additional service container alongside the actual test run, gets made reachable through its internal network name, and needs no persistent storage at all, since the entire email set is only relevant for the duration of a single pipeline run and can be discarded afterward.

This complete isolation per pipeline run also means parallel CI jobs don't interfere with each other through unexpected emails in the same mailbox, an advantage a shared, hosted test mail service doesn't readily offer without additional namespace separation.

To prevent parallel-running tests from interfering with each other through emails sent to the same fixed test address, it's advisable to use a unique, per-test-case generated recipient address, say by appending a random string or the test ID to a fixed local part, letting every test fetch exclusively its own emails and keeping test parallelization possible without mutual interference, even when several tests run the same registration or order process concurrently against the same test environment.

9. Limits of the approach: what a test SMTP server does not cover

A test SMTP server reliably checks what the application actually sends, but says nothing about how a real, public mail server will later handle that message, say whether SPF, DKIM, and DMARC records are configured correctly or whether the content gets rejected by the spam filters of major mail providers, aspects lying outside the actual application logic that need to be checked separately, say with dedicated deliverability services.

The table below compares the approaches presented for testing email delivery.

10. Summary

11. FAQ: Email Testing: The Essentials at a Glance