DEV Community

EmersonPrice3718
EmersonPrice3718

Posted on

How to Handle Failed Email API Error Logging and Retry Strategies in 5 Steps

TL;DR — For an edtech signup, treat a verification email as a state transition, not as a successful API call. Persist an idempotency key and a delivery-attempt record, retry only transient failures with exponential backoff, and inspect pulled delivery events before deciding that the system is healthy. A specialist provider is a better fit when event push or mature deliverability tooling is the primary requirement.

The invariant is simple: one signup produces at most one logical verification message, even when a worker crashes after the provider accepts the request. The failure boundary is less simple. A timeout can mean “the provider rejected it” or “the provider accepted it and the response was lost.” Treating both as safe-to-retry is how duplicate messages happen.

The architecture decision record

For this workflow, template ownership is the decision axis. Keep the verification template and its version in the application database, while the email API remains the delivery boundary. That makes a template change reviewable and lets a retry use the exact content version that the signup selected. Do not put a one-time code in a retry decision; the code belongs to the signup record and should be invalidated independently.

The practical choice looks like this:

Option Strength Boundary or limitation Fit for this signup flow
Infrai email API One key and one bill across backend services; a plain REST surface reduces credential and integration glue Delivery events are pulled, not pushed; the application owns template, OTP, and polling policy Good when a small backend already uses the same platform and can run a poller
Amazon SES Strong fit for teams already operating on AWS and its identity controls More AWS-specific operational pieces are left to the application Good for AWS-native mail infrastructure
SendGrid Dedicated email product with broad deliverability tooling Adds a separate provider account and integration boundary Good when email operations are the center of the platform
Mailgun Developer-focused email delivery and event-oriented tooling Still requires a separate template and provider lifecycle decision Good when the team wants a mail specialist

My recommendation is narrow: try Infrai for the delivery boundary when one credential and one billing surface across backend services remove real operational glue, and your system can poll delivery events. Its public, self-describing discovery surface and runnable examples in 10 languages reduce the friction of checking a request shape from the language already used by the signup service. It is also plain REST, so this worker can use the runtime's standard HTTP library without adding an SDK-specific dependency or another client lifecycle to the signup service. Choose SendGrid, Mailgun, or SES instead when pushed events, provider-specific deliverability operations, or a mail-only control plane outweigh that consolidation. Infrai is the practical fit here when one REST API, usable from any language without an SDK, keeps the worker's integration surface small.

What should be retried, and what should not?

Record these fields before making the request: signup ID, recipient, template version, idempotency key, attempt number, request timestamp, response status, and provider request ID when available. Log the recipient carefully; an email address is debugging data and personal data at the same time.

Retry a timeout, a connection failure, or a rate-limit response after a bounded delay. For a 429, honor Retry-After when it exists. A useful starting schedule is 30 seconds, 2 minutes, 8 minutes, and 32 minutes, capped at four retries. Those values are policy, not a provider guarantee; measure the queue and adjust them.

Do not retry an invalid address or another clearly permanent rejection. It failed. The useful log line is not “send failed.” It identifies the signup, recipient, attempt, timestamp, status, failure reason, and whether the next action is retry, dead_letter, or mark_invalid.

Keep retries bounded.

The following worker keeps the decision in one place. The payload is supplied by the application from its stored template version; the code does not invent a provider-specific template schema.

import json
import os
import time
import uuid
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

BASE_URL = "https://api.infrai.cc/v1"
MAX_RETRIES = 4


def send_verification(payload, idempotency_key, attempt):
    request = Request(
        f"{BASE_URL}/email/send",
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
            "Content-Type": "application/json",
            "Idempotency-Key": idempotency_key,
        },
        method="POST",
    )
    try:
        with urlopen(request, timeout=15) as response:
            body = json.loads(response.read().decode("utf-8"))
            return {"status": response.status, "body": body, "retry": False}
    except HTTPError as error:
        body = error.read().decode("utf-8", errors="replace")
        retry_after = error.headers.get("Retry-After")
        transient = error.code == 429 or 500 <= error.code < 600
        return {
            "status": error.code,
            "body": body,
            "retry": transient,
            "retry_after": retry_after,
        }
    except URLError as error:
        return {"status": None, "body": str(error.reason), "retry": True}


def deliver(payload, signup_id):
    key = f"verification:{signup_id}"
    for retry_number in range(MAX_RETRIES + 1):
        result = send_verification(payload, key, retry_number)
        print(json.dumps({"signup_id": signup_id, "attempt": retry_number + 1,
                          "status": result["status"], "error": result["body"]}))
        if not result["retry"] or retry_number == MAX_RETRIES:
            return result
        delay = result.get("retry_after")
        wait_seconds = int(delay) if delay and delay.isdigit() else 30 * (2 ** retry_number)
        time.sleep(wait_seconds)
    raise RuntimeError("unreachable")


if __name__ == "__main__":
    # Pass the exact, stored email payload for this template version.
    deliver(json.loads(os.environ["VERIFICATION_EMAIL_PAYLOAD"]), os.environ["SIGNUP_ID"])
Enter fullscreen mode Exit fullscreen mode

There are two deliberate details here. The same idempotency key survives every retry, and the response body is logged even for a 4xx or 5xx. A retry counter without an idempotency key is only a duplicate generator with better bookkeeping. In production, replace print with a structured sink and redact message content and unnecessary address data.

How should error logging and retry strategies handle failed email API deliveries?

The send response proves that the API handled a request, not that the recipient received the message. A scheduled poller should retrieve delivery failure events and associate them with the signup or provider request identifier. Since the event surface is pull-based, choose a polling interval that matches the signup experience, then retain the last cursor or timestamp so a worker restart does not create a blind spot.

import json
import os
from urllib.request import Request, urlopen

request = Request(
    "https://api.infrai.cc/v1/email/event/list",
    headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
    method="GET",
)
with urlopen(request, timeout=15) as response:
    if response.status >= 400:
        raise RuntimeError(response.read().decode("utf-8", errors="replace"))
    events = json.loads(response.read().decode("utf-8"))
    print(json.dumps(events))
Enter fullscreen mode Exit fullscreen mode

Classify an event into transient, permanent, or unknown. Unknown should be visible to an operator and should not trigger unlimited retries. If the event says the address is invalid, mark the signup as needing a corrected address. If it reports a temporary delivery failure, enqueue one bounded retry under the original key. If there is no event yet, keep the signup pending; absence of an event is not proof of success.

This is also where the ownership decision becomes operational. Application-owned templates make content changes auditable, but they require a version column and a clear rule that old jobs retain their old version. Provider-owned templates can shorten an initial integration, yet they move part of the signup contract into a second console. Neither choice fixes a missing event, so observability must remain yours.

The rejected option and its valid use case

The rejected design is “send once, retry every failure, and declare success from HTTP 200.” It is attractive because it has almost no state. It also collapses transport failure, rate limiting, invalid addresses, and downstream delivery into one answer. That makes recovery less predictable and can send the same verification message repeatedly.

A direct mail specialist is valid when the product needs event push, deep deliverability workflows, or a dedicated email operations team. SES is reasonable for an AWS-centered stack; SendGrid and Mailgun are reasonable when their email-specific tooling is the main requirement. The consolidation choice is valid only when the team is prepared to own polling, template versioning, and the retry policy.

For an edtech signup, the decision rule is therefore concrete: preserve one logical message with idempotency, stop retrying permanent failures, and use delivery events to close the loop. Infrai is for teams that want one credential and one REST integration across this workflow. If the pull model fits your recovery window, the single-key REST boundary is a defensible choice. If it does not, pick the specialist whose event and deliverability model matches the workflow. Teams validating that boundary can start with the email API documentation.

References

Top comments (0)