DEV Community

Francisco Perez
Francisco Perez

Posted on • Originally published at uncorreotemporal.com

How to Test Email Signup Flows in CI/CD Pipelines

#testing #python #automation #cicd

The Pain Every CI Pipeline Knows

It works locally. The signup form submits, the confirmation email arrives in your inbox, you paste the OTP, it verifies. Ship it.

Then CI runs. The test sends the signup request, polls... and times out. Or worse: it passes because you mocked the email sender and never actually tested that the email arrives, renders correctly, or contains a working code.

Email signup flows break in CI for a handful of predictable reasons:

  • Shared inboxes — two parallel test runs both register and poll the same address. Whichever test gets the email first passes; the other times out.
  • Mocked senders — you assert that send_email() was called, not that an email was received. Template bugs, SMTP credential expiry, and malformed links all pass through silently.
  • Brittle fixtures — hardcoded email addresses that accumulate stale messages across runs.
  • No isolation — there's no API to create a fresh inbox per test run, so you bolt on a shared test account and cross your fingers.

The fix is not a better mock. It's a real, isolated, programmable inbox that your test owns for its lifetime.

Quick Start

All examples in this article are from the temporary-email-api-examples repo. The python-example/ folder is a single-file script that demonstrates the full flow with plain requests — no extra libraries required.

Set two environment variables before running:

export UCT_API_BASE_URL="https://uncorreotemporal.com/api/v1"
export UCT_EMAIL_API_KEY="your_api_key_here"
Enter fullscreen mode Exit fullscreen mode

Then:

git clone https://github.com/francofuji/temporary-email-api-examples
cd temporary-email-api-examples/python-example
pip install requests
python main.py
Enter fullscreen mode Exit fullscreen mode

You will see a temporary inbox created, a polling loop start, and — once you trigger a real email to the address — an OTP extracted and printed.

The Flow: Create Inbox - Wait for Email - Extract OTP

The pattern is three steps. Here is how the example implements each one.

1. Create the Inbox 

import os, requests

API_BASE_URL = os.getenv("UCT_API_BASE_URL")
API_KEY = os.getenv("UCT_EMAIL_API_KEY")
HEADERS = {"X-API-Key": API_KEY}

create_resp = requests.post(f"{API_BASE_URL}/inboxes", headers=HEADERS, timeout=10)
create_resp.raise_for_status()

inbox = create_resp.json()
inbox_id = inbox.get("id")
Enter fullscreen mode Exit fullscreen mode

One POST, one inbox. No signup flow, no shared state. The inbox_id is the key you use for the next two steps.

2. Poll for Messages 

import time, re

pattern = re.compile(r"\b(\d{6})\b")
start_time = time.time()

while True:
    if time.time() - start_time > 60:
        raise SystemExit("Timed out waiting for OTP")

    try:
        msg_resp = requests.get(
            f"{API_BASE_URL}/inboxes/{inbox_id}/messages",
            headers=HEADERS,
            timeout=10,
        )
        if msg_resp.status_code >= 500:
            time.sleep(3)
            continue
        msg_resp.raise_for_status()
        messages = msg_resp.json()
    except requests.RequestException as exc:
        print(f"Request failed: {exc}. Retrying...")
        time.sleep(3)
        continue

    if not messages:
        time.sleep(3)
        continue
Enter fullscreen mode Exit fullscreen mode

The loop handles three distinct cases: server errors (5xx, which you retry), request failures (network blips), and an empty inbox (normal while waiting). Separating these is what makes polling reliable in CI — you don't bail on transient failures.

3. Extract the OTP 

    body = messages[0].get("body", "")
    match = pattern.search(body)
    if match:
        otp = match.group(1)
        print(f"OTP found: {otp}")
        break
Enter fullscreen mode Exit fullscreen mode

The regex \b(\d{6})\b matches any standalone 6-digit sequence. Adjust the pattern to match your application's format.

CI/CD Tips

Set an explicit timeout and fail loudly. The example uses 60 seconds. In CI, an infinite loop or a silently hanging test is worse than a hard failure. If the email doesn't arrive in 60 seconds, something upstream is broken — raise, don't sleep forever.

Use fixed poll intervals, not adaptive backoff. Poll every 3 seconds. Predictable timing makes failures easier to diagnose.

Handle 5xx separately from 4xx. A 500 from the messages endpoint is probably transient. A 401 or 404 means your credentials or inbox ID are wrong — retrying won't help. The example retries 5xx and raises immediately on 4xx via raise_for_status().

Don't share inbox IDs across test cases. Create one inbox per test run.

Store credentials in CI secrets, not in code.

# GitHub Actions
env:
  UCT_API_BASE_URL: ${{ vars.UCT_API_BASE_URL }}
  UCT_EMAIL_API_KEY: ${{ secrets.UCT_EMAIL_API_KEY }}
Enter fullscreen mode Exit fullscreen mode

Why This Works in CI

Most email testing approaches fail in CI because they depend on something external: a shared inbox, a stubbed SMTP server, or a hardcoded test account that accumulates state across runs.

This approach is stateless from the pipeline's perspective. Each run calls POST /inboxes, gets a fresh address and ID back, and owns that inbox exclusively until the test exits or the TTL expires. There's nothing to clean up, nothing to coordinate between parallel jobs, and no leftover messages from a previous run to confuse the polling logic.

The inbox also receives real SMTP delivery — the same delivery path your production email goes through. If your email provider is down, your template is broken, or your SMTP credentials expired, the test will fail. That's the point. A test that can't catch those failures isn't testing email; it's testing whether your mock function was called.

Conclusion

Email signup testing is broken in most CI pipelines not because it's technically hard but because the default tools — mocks, shared inboxes, SMTP dev servers — don't give you the properties you need: isolation, real delivery, and an API to read the result programmatically.

The pattern in this article is straightforward: create a dedicated inbox per test, poll for the message with a hard timeout, extract the OTP or link, and proceed. The Python example is a working implementation you can drop into any pipeline today.

Next steps:

Top comments (0)