Your Playwright test fills the signup form. The app sends the confirmation email. Mailpit shows the message with the right subject and link. The test passes.
Then staging sends the same email to a real address and nothing arrives.
The local catcher did its job. It proved the app opened an SMTP connection, built a MIME message, and rendered the template. It did not prove the message could reach an inbox.
What the catcher checks
Mailpit, MailHog, and similar tools are useful. They are fast, free, and easy to run in CI. They catch template errors, missing links, broken HTML, and the wrong recipient in the test payload. Keep them.
The problem starts when the local catcher is the only email test. It sits inside your network, so it never exercises the parts that decide whether a real message arrives.
What it cannot see
| Failure | What the local catcher shows | What the real inbox shows |
|---|---|---|
| DNS or MX is wrong | The message is delivered locally | The message bounces or disappears |
| DKIM key was rotated | The message is signed with the old key | The message fails authentication |
| SPF or DMARC is misaligned | The message is accepted | The message is filtered or rejected |
| The provider is sandboxed | The message is delivered | Only allowlisted recipients receive it |
| Spam filtering rewrites a link | The link is intact | The link points somewhere else |
| The recipient is over quota | The message is delivered | The message is deferred or dropped |
None of those failures need a code change to reproduce. They need a real address outside your network.
The pattern: one inbox per test run
Create a real inbox for the test, use its address in the form, wait for the message, assert its contents, then delete the inbox. The inbox can be a hosted service or a small internal service on your own domain. The test shape is the same.
const inbox = await inboxApi.create({ ttl: "15m" });
await page.fill("#email", inbox.address);
await page.click('button[type="submit"]');
const message = await inbox.waitForMessage({
subject: /confirm/i,
timeout: 30_000,
});
const link = message.links.find((url) => url.host === "app.example.com");
await page.goto(link);
await expect(page.getByText("Welcome")).toBeVisible();
The code assumes a small helper; the shape is the point, not the package. The helper is the part you build or buy. The assertions are the part that catches bugs:
- The recipient is the exact per-run address, not a shared inbox.
- The sender domain is yours.
- SPF, DKIM, and DMARC pass.
- The link host is on an allowlist. Never follow arbitrary links from email.
- The OTP format and expiry match the product.
- The message contains no real user data.
A few operational rules keep the test reliable:
- One inbox per worker. A shared inbox gives parallel tests the same message.
- Set a TTL and delete the inbox in teardown.
- Log the message id, not the body.
- Do not use a shared Gmail account. MFA, rate limits, and someone reading the inbox will break the test.
Keep the fast test
Run the local catcher on every commit. Run one real-inbox test on the staging deploy or nightly. The first catches template mistakes in seconds. The second catches the failures that only appear after the message leaves your network.
Replacing Mailpit with a real inbox makes the suite slower and more expensive. Replacing the real-inbox test with Mailpit makes the suite confident about the wrong thing.
Try one this week
If your suite only asserts against Mailpit, add one real-inbox test. Pick the signup email or the password reset, because those are the flows users notice first. Run it against staging once. If it passes, keep it in the nightly suite. If it fails, you found a bug your local tests could not see.
I work on Emcognito. We are testing a hosted version of this: a real address, an HTTP read, and a signed webhook. It is a waitlist, not a live product. If you run email tests, the page asks what you would pay: https://emcognito.com/test-inbox?src=devto. No card, and the article works without it.
Top comments (0)