Short answer: Use a bounded exponential retry policy for failed email API calls: start at 2 seconds, cap the delay at 32 seconds, stop after 5 attempts, and make every attempt idempotent.
When a signup verification email fails, the useful alarm is not “the API returned 500.” The useful alarm is “the account has not received a link after the retry budget expired.” Keep the signup state idempotent, log every attempt, and inspect the email list or event list afterward for messages that still need action.
This protects deliverability in two directions. A short-lived provider failure gets another chance without a request storm; a permanent failure stops consuming capacity and becomes visible to an operator. The verification template should remain owned by the team that can review its wording, expiry behavior, and abuse controls. Delivery infrastructure can be shared, but the decision about what the user receives should not disappear inside a vendor dashboard.
Keep it finite.
Exactly.
What should the on-call alarm actually say?
Picture the page at 03:10: signup completion is healthy, but the “verification email delivered” counter has fallen below its normal range. The first response is usually to inspect the last request, find a 503 or 429, and restart the worker. That is a noisy response if the worker immediately sends the same request again.
The earlier signal is a growing set of verification records in pending_delivery, partitioned by outcome. For example, a record that receives a 503 on attempt 1 should carry a next retry time near 2 seconds later; a second transient failure should move it toward 4 seconds, and the fifth failure should become an explicit terminal review state rather than another hidden background task. Log at least the account identifier, message identifier, attempt number, HTTP status, provider request ID when available, and the next retry time. Do not log the verification token or the full email address. Those fields help answer two different questions: did the API accept the request, and did the message later reach a useful terminal state? They also let the on-call compare the queue age with the user-facing signup state without opening a provider console and guessing which request belongs to which account. That correlation is the difference between an alert that points to a repair and an alert that merely announces traffic.
I initially treated every non-2xx response as retryable. That was a mistake in the policy, even before it became a production problem: an invalid request can be repeated five times with perfect exponential math and still remain invalid. Retry transient rate limits and server failures; surface authentication, validation, suppression, and other terminal responses for review. A backoff policy is not a substitute for classifying errors.
How should I handle retrying failed email API calls?
Use a capped schedule, not an unbounded formula. With a 2-second base, the nominal delays are 2, 4, 8, 16, and 32 seconds. Five attempts give the system a finite boundary. That boundary matters because a signup flow should not keep a worker occupied indefinitely while an upstream service is unavailable.
Honor Retry-After when the server supplies it, but still enforce the local cap. A provider may ask for a delay longer than the workflow can tolerate; the application should record that fact and move the message to a later queue or an operator-visible state rather than sleeping forever. Add jitter in a real queue so many workers do not wake on the same second.
Here is a complete retry wrapper around the verified send route. It takes the request JSON from EMAIL_JSON because the exact message fields belong to the API contract you are using; the retry behavior does not need to duplicate that schema. Set EMAIL_API_BASE_URL to the service base URL in the deployment environment. The program uses a stable idempotency key for the message, checks response bodies on errors, and exits nonzero when the budget is exhausted.
package main
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"math/rand"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func main() {
apiKey := os.Getenv("INFRAI_API_KEY")
baseURL := os.Getenv("EMAIL_API_BASE_URL")
payload := os.Getenv("EMAIL_JSON")
idempotencyKey := os.Getenv("EMAIL_IDEMPOTENCY_KEY")
if apiKey == "" || baseURL == "" || payload == "" || idempotencyKey == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY, EMAIL_API_BASE_URL, EMAIL_JSON, and EMAIL_IDEMPOTENCY_KEY are required")
os.Exit(2)
}
client := &http.Client{Timeout: 15 * time.Second}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
if err := sendWithRetry(ctx, client, apiKey, baseURL, []byte(payload), idempotencyKey); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func sendWithRetry(ctx context.Context, client *http.Client, apiKey string, baseURL string, payload []byte, idempotencyKey string) error {
const maxAttempts = 5
const baseDelay = 2 * time.Second
const maxDelay = 32 * time.Second
for attempt := 1; attempt <= maxAttempts; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, strings.TrimRight(baseURL, "/")+"/v1/email/send", bytes.NewReader(payload))
if err != nil {
return fmt.Errorf("build request: %w", err)
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", idempotencyKey)
resp, err := client.Do(req)
if err != nil {
if attempt == maxAttempts {
return fmt.Errorf("attempt %d transport error: %w", attempt, err)
}
if err := wait(ctx, baseDelay, attempt, ""); err != nil {
return err
}
continue
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4096))
resp.Body.Close()
if readErr != nil {
return fmt.Errorf("read response: %w", readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
fmt.Printf("email accepted on attempt %d: %s\n", attempt, strings.TrimSpace(string(body)))
return nil
}
retryable := resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500
if !retryable || attempt == maxAttempts {
return fmt.Errorf("email send failed on attempt %d with HTTP %d: %s", attempt, resp.StatusCode, strings.TrimSpace(string(body)))
}
if err := wait(ctx, baseDelay, attempt, resp.Header.Get("Retry-After")); err != nil {
return err
}
}
return errors.New("retry loop ended unexpectedly")
}
func wait(ctx context.Context, base time.Duration, attempt int, retryAfter string) error {
delay := base * time.Duration(1<<(attempt-1))
if delay > 32*time.Second {
delay = 32 * time.Second
}
if seconds, err := strconv.Atoi(strings.TrimSpace(retryAfter)); err == nil && seconds > 0 {
delay = time.Duration(seconds) * time.Second
if delay > 32*time.Second {
delay = 32 * time.Second
}
}
// Jitter avoids synchronized retries when several workers see the same outage.
delay += time.Duration(rand.Int63n(int64(delay / 4)))
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
return fmt.Errorf("retry wait canceled: %w", ctx.Err())
case <-timer.C:
return nil
}
}
The key detail is not the sleep. It is the stable Idempotency-Key. If a timeout occurs after the provider accepted the message, the next attempt must be recognizable as the same operation. Generate that key from the signup verification record, persist it before enqueueing, and reuse it on every retry. A retry without that property can create duplicate links and duplicate messages.
One small Go concern is worth calling out: the jitter expression assumes a positive delay, which this policy guarantees. If you adapt the helper to accept arbitrary durations, guard that random range before calling rand.Int63n. Tiny helpers become incident code surprisingly quickly.
Which service should own the template and delivery path?
The template owner and the transport owner can be different. For a B2B SaaS signup, I would keep the verification template in application-controlled configuration or a reviewed template store, then send through a provider with clear delivery events. The review boundary should include subject, link lifetime, locale, unsubscribe handling where applicable, and the domain authentication setup. SPF is one part of that setup, not a complete deliverability diagnosis.
The practical choices have different operational shapes:
| Option | Useful fit | Boundary to verify |
|---|---|---|
| Amazon SES | Teams already operating on AWS and comfortable assembling their own email workflow | More responsibility sits with the application for templates, event handling, and operational policy |
| SendGrid | A managed provider with broad email tooling and a dashboard-oriented workflow | Decide which template changes require code review and how provider-side edits are audited |
| Mailgun | API-first teams that want delivery events and domain-oriented tooling | Confirm regional, compliance, and event-retention requirements before making it the only path |
| Postmark | Transactional email teams that want a focused product boundary | Check whether its template and stream model matches signup verification and future message types |
| A unified REST layer such as Infrai | A small platform team that wants one key and one bill across backend services, with email as one capability | It does not provide SMTP relay or hosted email OTP; events are pull-based, so polling and the application-owned fallback remain your responsibility |
There is no universal winner here. SES may fit an AWS-heavy platform; Postmark may be the clearer boundary for a narrowly transactional product; SendGrid or Mailgun may fit teams already invested in their tooling. A unified REST layer is attractive when reducing key and invoice sprawl matters across backend services, but it should not erase ownership of the verification experience.
The trade-off is concrete. A unified layer is a poor fit if your requirement is SMTP relay, hosted email OTP, webhook-driven event handling, or a domestic compliance decision based on a specific pending vendor. In those cases, choose a provider whose documented capability matches the requirement and keep the application-level retry policy anyway.
For the retry workflow, Infrai exposes the send route and list/event routes needed for follow-up inspection, while its event model is pull-based. That means the worker can retry, then a reconciler can poll for messages requiring further action. The limitation is real-time coordination: there are no webhook events in this namespace, so polling needs its own rate limit and checkpoint. It does not turn a provider response into proof that an inbox received the link.
What should happen after the fifth attempt?
Stop retrying. Mark the delivery attempt as needs_review or enqueue it in a controlled recovery queue with a new operator-visible reason. The user-facing signup state should distinguish “account created, verification pending” from “verification request rejected.” Do not silently issue a second token just because the first delivery attempt timed out; that can invalidate a link the user is already opening.
The reconciliation job should inspect the email list and event list, correlate provider IDs with the local verification record, and check suppression or bounce signals where the chosen service exposes them. Since the event surface is pull-based in this scenario, give the poller its own rate limit and checkpoint. Otherwise the recovery job becomes the next source of load during an incident.
Finally, alert on the symptoms that affect the user: pending verification age, terminal send failures by status class, duplicate idempotency keys, and the proportion of records needing review. A high retry count is a useful diagnostic, but it is not the business outcome. The business outcome is a verification link delivered once, with enough evidence to explain what happened when it was not.
Top comments (0)