AI Agent Email Probes Need a Test Contract

AI agents are good at following a workflow such as “sign up, open the verification email, and finish setup.” They are much less good at dealing with an inbox that is slow, empty, duplicated, or shared with another test.

That is why email automation needs a small test contract. A temp mailbox is only one capability in the workflow. The agent also needs to know who owns the mailbox, how long to poll, what counts as evidence, and when to stop.

The problem: seeing an email is not the same as proving the workflow

A typical agent-driven test looks simple:

1. Create a mailbox.
2. Ask the application to send a verification message.
3. Poll for a message.
4. Extract a link or code.
5. Continue the workflow.

The hidden failure modes are less simple. The message may belong to an earlier run. The agent may click a link from a different account. A retry may create two valid messages. Or the inbox provider may return a partial result while delivery is still happening.

Without explicit boundaries, the agent can make a plausible choice and still produce a false positive. This is especially common when a free temporary email workflow is reused across parallel CI jobs.

The useful mental model is: an email probe is a bounded observation, not a magic “wait until it works” action.

Define the mailbox contract

Before the agent starts, create a record with the minimum data needed to keep the run isolated:

{
  "run_id": "signup-1842",
  "mailbox": "[e-mail]",
  "created_at": "2026-10-31T08:00:00Z",
  "expires_at": "2026-10-31T08:05:00Z",
  "expected_recipient": "[e-mail]",
  "seen_message_ids": []
}

The exact provider does not matter as much as the contract. The run_id should be carried into the test logs, and the expected recipient should be checked against every candidate message. If the system supports a correlation token in the subject or body, include that too.

This is the same design direction as an idempotent contract for AI agent email tools: repeated observations should not create repeated side effects.

Model polling as a bounded probe

Polling needs a deadline and a clear result. A basic state machine is enough:

CREATED -> WAITING -> FOUND -> VERIFIED
                    \-> EXPIRED
                    \-> INVALID

On each poll, fetch only messages newer than the mailbox creation time. Discard messages sent to a different recipient. Ignore message IDs already recorded. If the message has the right recipient and correlation token, move to FOUND and save the message ID before extracting the link.

The important detail is the deadline. An AI agent should never keep polling because the next attempt “might” work. Give it a small retry budget, such as a fixed number of probes inside a fixed time window. When the budget ends, return EXPIRED with evidence. That result is much easier to debug than a vague timeout.

For a practical reference, the ideas in passwordless email testing without inbox chaos are useful when designing isolation for concurrent test workers.

Record evidence, not just success

An agent needs structured evidence so it can decide what to do next without guessing. A successful probe might return:

{
  "status": "found",
  "message_id": "msg-901",
  "recipient": "[e-mail]",
  "received_at": "2026-10-31T08:01:14Z",
  "age_seconds": 14,
  "correlation_match": true
}

Keep the raw message out of normal logs when it contains tokens or personal data. Store a redacted excerpt or a content hash instead. The agent usually needs to know that the right message arrived, not to repeat a secret into a transcript.

If your logs say only email found, debugging becomes realy hard. If they say which mailbox, message ID, recipient, age, and match decision were used, the failure is much more obvious.

A small implementation sketch

Here is intentionally plain pseudocode for the probe:

deadline = clock.now() + timeout
seen = set()

while clock.now() < deadline:
    messages = inbox.list(after=mailbox.created_at)
    for message in messages:
        if message.id in seen:
            continue
        seen.add(message.id)

        if message.recipient != mailbox.expected_recipient:
            continue
        if not matches_correlation(message, run_id):
            continue

        return Evidence(status="found", message_id=message.id)

    sleep(backoff.next())

return Evidence(status="expired", attempts=len(seen))

The probe does not click a link or mutate application state. That separation is usefull: one component observes, another component performs the next action, and the agent can explain which evidence allowed it to proceed.

Keep cleanup and isolation explicit

Cleanup should be part of the workflow contract, not an afterthought. Mark the mailbox as closed after VERIFIED, INVALID, or EXPIRED. In CI, also include cleanup in a finally block so a failed assertion does not leave a mailbox that a later worker might reuse.

When a provider cannot guarantee unique inboxes, add an application-level lease. Store the mailbox and run ID together, reject a lease that is already active, and make lease release idempotent. This protects the test even when the provider behaves a bit unexpected.

Searchers sometimes type phrases such as “tamp mail com” or “tepm mail com” when looking for test mail. Those strings can help explain real support traffic, but they should never be used as identifiers or matching rules in the test itself.

Final checklist

• Does every mailbox have a unique run ID?
• Is the expected recipient checked on every message?
• Are old message IDs ignored?
• Is polling bounded by both time and attempts?
• Does the result include evidence and a reason for stopping?
• Are tokens redacted from ordinary logs?
• Is cleanup safe to run more than once?

The goal is not to make an AI agent wait more patiently. It is to give the agent a small, observable contract. Once mailbox ownership, timing, evidence, and cleanup are explicit, email stops being a flaky mystery and becomes another testable developer tool.