Testing email in CI: end-to-end tests for signup and password reset emails

What a mocked mailer can't tell you

Most test suites replace the mailer with a fake and assert that it was called with the right recipient and template. That is a good unit test, and you should keep it. But the flows that depend on email - sign-up verification, password reset, magic-link sign-in, invitations - break in ways a fake never sees:

• the link points at localhost or an old domain because the base URL is configured differently in staging,
• the token in the link is double-encoded, truncated, or expires before the email arrives,
• the background worker that sends mail isn't running in the deployed environment,
• the SMTP credentials or API key in that environment are wrong or revoked,
• the template renders in development but fails with production data (a missing name, a long product title),
• the message is sent but rejected or filtered as spam because the sending domain isn't authenticated.

Each of these reaches users as "I never got the email". An end-to-end test that waits for a real message and clicks the real link catches all of them, which is why the critical email flows deserve at least one.

Three levels of email testing

Use all three, in proportion: many unit tests, integration tests for every template, and a handful of end-to-end tests for the flows that lock users out when they break.

The pattern: trigger, wait, extract, act

1. Note the time, then trigger the email the way a user would - fill in the sign-up form, request a password reset.
2. Wait for the message in the test inbox, polling until it arrives or a deadline passes.
3. Extract what the user would use: the verification link, the one-time code.
4. Act and assert: open the link, enter the code, and check the application reached the right state.

With AnyMail the test inbox is an ordinary inbox on your own domain, and each of the official SDKs has a waitForMessage call made for exactly this step. Here is a Playwright test in TypeScript:

import { expect, test } from '@playwright/test';
import { Anymail } from '@anymail/sdk';

test.beforeAll(async () => {
  await Anymail.connect({ apiKey: process.env.ANYMAIL_API_KEY! });
});

/** The first link in an email body that contains `path` - works for text and HTML bodies. */
function findLink(body: string, path: string): string {
  const links = [...body.matchAll(/https?:\/\/[^\s"'<>]+/g)].map((match) => match[0].replaceAll('&amp;', '&'));
  const link = links.find((candidate) => candidate.includes(path));
  if (!link) throw new Error(`No link containing ${path} in the email`);
  return link;
}

test('a new account can verify its email address', async ({ page }, testInfo) => {
  // One inbox per parallel worker, so two workers never read each other's mail.
  const inbox = `e2e-${testInfo.parallelIndex}@example.com`;
  const runId = crypto.randomUUID().slice(0, 8);
  const startedAt = new Date(Date.now() - 10_000); // tolerate a few seconds of clock skew
  await deleteTestUser(inbox); // your app's own helper: remove an account left over from an earlier run

  await page.goto('/register');
  await page.getByLabel('Name').fill(`E2E ${runId}`);
  await page.getByLabel('Email').fill(inbox);
  await page.getByLabel('Password').fill(`pw-${runId}-${Date.now()}`);
  await page.getByRole('button', { name: 'Create account' }).click();

  const mail = await Anymail.waitForMessage({
    inbox,
    subject: 'Verify your email',
    contains: runId, // the greeting includes the name we registered with
    since: startedAt,
    timeoutMs: 60_000
  });

  await page.goto(findLink(mail.body, '/verify-email'));
  await expect(page.getByText('Your email address is verified')).toBeVisible();

  await mail.moveToTrash();
});

The same idea for a password reset, in Python with pytest and Playwright:

import re
from datetime import datetime, timedelta, timezone

import pytest
from anymail_sdk import Anymail
from playwright.sync_api import Page, expect

APP_URL = "https://staging.example.com"
INBOX = "e2e-reset@example.com"
RESET_LINK = re.compile(r"https://[^\s\"'<>]+/reset-password[^\s\"'<>]*")


@pytest.fixture(scope="session", autouse=True)
def anymail():
    return Anymail.connect()  # ANYMAIL_API_KEY comes from a CI secret


def test_password_reset_email(page: Page):
    started_at = datetime.now(timezone.utc) - timedelta(seconds=10)

    page.goto(f"{APP_URL}/forgot-password")
    page.get_by_label("Email").fill(INBOX)
    page.get_by_role("button", name="Send reset link").click()

    mail = Anymail.wait_for_message(inbox=INBOX, subject="Reset your password", since=started_at, timeout=60)
    match = RESET_LINK.search(mail.body)
    assert match, f"no reset link in: {mail.body}"

    page.goto(match.group().replace("&amp;", "&"))  # HTML bodies escape &
    page.get_by_label("New password").fill("correct-horse-battery-staple")
    page.get_by_role("button", name="Set new password").click()
    expect(page.get_by_text("Your password has been changed")).to_be_visible()

    mail.move_to_trash()

And in Go, for a backend test that triggers the email through your API instead of a browser:

var resetLink = regexp.MustCompile(`https://[^\s"'<>]+/reset-password[^\s"'<>]*`)

func TestPasswordResetEmail(t *testing.T) {
	ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
	defer cancel()

	mail, err := anymailsdk.Connect(ctx, anymailsdk.Options{APIKey: os.Getenv("ANYMAIL_API_KEY")})
	if err != nil {
		t.Fatal(err)
	}
	const inbox = "e2e-reset@example.com"
	startedAt := time.Now().Add(-10 * time.Second)

	if err := triggerPasswordReset(inbox); err != nil { // your app, e.g. POST /forgot-password
		t.Fatal(err)
	}

	msg, err := mail.WaitForMessage(ctx, anymailsdk.WaitOptions{
		Inbox:   inbox,
		Subject: "Reset your password",
		Since:   startedAt,
		Timeout: time.Minute,
	})
	if errors.Is(err, anymailsdk.ErrWaitTimeout) {
		t.Fatal("no password reset email arrived within a minute")
	}
	if err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { _ = msg.MoveToTrash(context.Background()) })

	link := strings.ReplaceAll(resetLink.FindString(msg.Body), "&amp;", "&") // HTML bodies escape &
	if link == "" {
		t.Fatalf("no reset link in: %s", msg.Body)
	}
	resp, err := http.Get(link)
	if err != nil {
		t.Fatal(err)
	}
	resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		t.Fatalf("reset link returned %d", resp.StatusCode)
	}
}

All three read the API key from an environment variable, which in CI is a secret. API access is part of AnyMail's Pro plan; the API reference lists every call the SDKs make.

Waiting correctly

Poll with a deadline, never sleep

sleep(10) followed by "read the inbox" is the classic source of flaky email tests: too short on a slow day, wasted time on a fast one. Poll instead and stop as soon as the message is there. waitForMessage polls every two seconds by default, backs off when the API rate-limits it, and fails with a clear timeout error (WaitTimeoutError in TypeScript and Python, ErrWaitTimeout in Go) when the deadline passes.

Don't miss a fast email

By default waitForMessage only accepts messages that arrive after it starts, so it can't pick up yesterday's email by mistake. That creates a race: if the application is quick and the email lands before the wait begins, it's ignored. The samples avoid it by noting the time before triggering the email and passing it as since. Subtract a few seconds to allow for the difference between the CI machine's clock and the server's.

Choosing the timeout

Mail between two inboxes on the same provider arrives in seconds; mail that crosses the internet usually does too, but not always. One cause of minutes-long delays is greylisting: the receiving server deliberately turns away the first message from an unknown sending network and accepts the retry a few minutes later. AnyMail greylists incoming internet mail - when your application sends through another provider to an AnyMail test inbox, the first message from that provider's network to that inbox is accepted only on a retry at least five minutes later. After that the pair is remembered, and a network that has passed a handful of times skips the check entirely. Mail your application sends through AnyMail itself to an AnyMail inbox is delivered internally and never greylisted.

A 60-second wait is a sensible default for flows whose mail is delivered internally or from an already-known network. For a brand-new setup, run the test once with a long timeout to get past greylisting rather than raising every timeout permanently. Set the test framework's own timeout comfortably above the wait, so a timeout is reported as "no email arrived" rather than as a generic test failure.

Matching the right message

A shared test inbox receives mail from every run, branch and developer. Filter as tightly as you can:

• Subject and sender (subject, from) narrow it to the right kind of email.
• Something unique to this run (contains) makes it unambiguous. The sign-up sample registers with a name containing a random run id, and the greeting in the email includes the name. If your emails contain nothing user-supplied, match on a predicate instead (match in TypeScript and Python, Match in Go) - for example the token you can read from your test database.
• A start time (since) excludes everything older.

Extracting links and codes

Read the body your users read. Many applications send HTML with a plain-text alternative; AnyMail keeps the HTML part when there is one, so search the message's body rather than its text-only field, and decode & back to & in links taken from HTML - a link with an escaped ampersand loses its second query parameter. A narrow pattern for the path you expect is more robust than "the first link in the email", which is often the logo.

one-time code:    \b\d{6}\b
verification link: https://[^\s"'<>]+/verify-email[^\s"'<>]*

Isolation between tests and runs

AnyMail delivers only to inboxes you created and their aliases (up to 10 per inbox, added from the inbox's Aliasespanel), so you can't mint a fresh address per test the way plus-addressing or a catch-all allows; plan isolation around a small set of fixed test inboxes instead:

• One inbox per parallel worker. Create e2e-0@ to e2e-3@ for four workers and pick one from the worker index (testInfo.parallelIndex in Playwright), so two tests never wait in the same inbox at the same time.
• Reset state before, not after. A sign-up test needs the address to be free, so delete a leftover test account at the start of the test. Clean-up that runs at the end is skipped exactly when a test crashes.
• Tidy the inbox. Moving the message to Trash afterwards keeps the inbox small and fast to search; Trash empties itself after the retention period.
• Keep test mail away from real mail. An API key reaches every inbox on the account it belongs to, so a key stored in CI should not be able to read your production mailboxes. Put the test inboxes on a separate account if you can, and at least on their own mail server and test domain.

Keeping it reliable in CI

• Run email end-to-end tests against an environment that sends real mail - staging, or production with test accounts.
• On failure, log what you can: the inbox, the time window and, if a message arrived but didn't match, its subject and Message-ID. That turns "flaky" into a diagnosis.
• Allow at most one automatic retry. A test that only passes on its third attempt is telling you something about the email flow.
• If mail arrives but lands in spam at the real providers, the test inbox won't show that - check deliverability separately, with the headers of a message sent to Gmail or Outlook.com.

Local SMTP catchers such as Mailpit

For development and integration tests, a local SMTP catcher is often the better tool. Mailpit, for example, runs as a single binary or a Docker container (axllent/mailpit), accepts mail on SMTP port 1025 and shows it in a web interface on port 8025. Point your application's SMTP settings at it and nothing leaves your machine. It has a REST API for integration tests, can check links and HTML compatibility, can score spamminess through a SpamAssassin server, and has a "chaos" mode that answers with SMTP errors so you can test how your application copes with them.

Hosted email-testing services sit between the two and are worth a look if you need many throwaway addresses. Whichever you pick, the pattern above stays the same: trigger, wait with a deadline, extract, act - and to make the sending side just as dependable, see sending email from your application.