TL;DR: For SMS event notifications, don't treat resend failures or suspected carrier filtering as a reason to send again immediately. First prove that the request was accepted into your own system, that its sender identity and message class match the destination's approved policy, and that a provider produced a terminal outcome. Retry only transient transport failures, cap the attempt count, and preserve a correlation trail that explains each decision without storing the one-time code itself.
That rule works for telehealth login verification and for an edtech marketplace where a seller must enter an access code before viewing a new order. The latter is the running example here because it exposes the real operational tension: a delayed alert is inconvenient, but an uncontrolled resend loop can create duplicate messages, weaken abuse controls, and leave the support team with no defensible account of what happened.
The objective is not “deliver at any cost.” It is to meet a delivery SLO while retaining evidence that the system enforced its own policy.
How should SMS event failures change carrier-filtering decisions?
A resend creates a second event. If the first attempt is merely delayed, the seller can receive two valid-looking codes in reverse order; if both attempts share a mutable database row, the later write may also erase the evidence needed to understand the first. More traffic then reaches the same policy boundary that rejected or filtered the original message.
Classify the path before changing it. A useful timeline has four independent moments: the application accepted the challenge request, the dispatcher attempted a channel, the downstream transport acknowledged it, and a terminal delivery or failure event arrived. “The user did not receive it” is an observation, not a transport status.
For each attempt, record an opaque challenge ID, an attempt ID, the channel, a destination fingerprint, the sender-policy profile, timestamps, and a normalized outcome. Do not log the code. OWASP's forgot-password guidance says reset tokens or codes should be random, long enough to resist brute force, stored securely, single use, and expiring; it also calls for consistent responses and protections against excessive automated submissions. Those constraints apply cleanly to login verification even though the surrounding transaction differs.
I would use this first decision split:
| Evidence | Interpretation | Operator action |
|---|---|---|
| No durable challenge record | Failure is inside the application boundary | Fix admission or persistence; do not resend |
| Attempt exists, no transport acknowledgement | Dispatch is unknown or timed out | Reconcile by attempt ID before another send |
| Explicit policy or registration rejection | Configuration is not eligible for this route | Quarantine the route and correct its evidence |
| Temporary transport failure | Delivery may succeed later | Retry with bounded backoff and the same challenge policy |
| Accepted, no terminal event | Outcome is unknown | Wait to the defined deadline, then offer a controlled fallback |
| Terminal delivery, user still blocked | Delivery and authentication disagree | Investigate code state, expiry, clock, and abuse controls |
The distinction between “unknown” and “failed” matters. Unknown work is where duplicate sends breed.
Stop there.
Make policy eligibility a deployment input
Sender registration and signatures should not live as tribal knowledge in a messaging dashboard. Model them as deployment data keyed by destination region, channel, message class, and sender identity. The deployable artifact does not need to contain regulatory prose; it needs a reviewed assertion that the selected identity is eligible, a reference to the approval evidence, and effective dates so an expired approval cannot silently remain routable.
Do not infer eligibility from a country code alone. The application should ask a policy service for a route, and that service should either return a versioned profile or deny the send. This keeps US and EU behavior explicit without pretending that one global sender configuration has the same standing everywhere. Legal and carrier requirements can change outside a code release, so the evidence owner and review cadence belong in the control plane.
The same discipline applies to email fallback. Yahoo's sender guidance calls for authenticated mail, valid forward and reverse DNS, low complaint rates, easy unsubscribe for subscribed mail, and separation of marketing from transactional traffic. A login code is transactional, but switching channels does not erase sender-quality obligations. Treat fallback as another governed route, not an escape hatch.
For the seller-order flow, the policy decision can be represented by a small, provider-neutral contract:
package messaging
import "time"
type RouteRequest struct {
ChallengeID string
Region string
Channel string
MessageClass string
}
type RouteDecision struct {
Allowed bool
PolicyVersion string
SenderProfileID string
EvidenceRef string
ValidUntil time.Time
}
func Eligible(now time.Time, d RouteDecision) bool {
return d.Allowed &&
d.PolicyVersion != "" &&
d.SenderProfileID != "" &&
d.EvidenceRef != "" &&
now.Before(d.ValidUntil)
}
Rejecting an ineligible route locally is preferable to sending speculative traffic and interpreting a downstream rejection after the fact. It also produces a cleaner compliance record: policy version X denied attempt Y before message content left the boundary.
Retry the event, not the button press
An API handler should create or retrieve a challenge, then enqueue a uniquely identified attempt. The worker owns transport retries.
The browser doesn't.
package messaging
import (
"context"
"errors"
"time"
)
var ErrUnknownOutcome = errors.New("transport outcome unknown")
type Attempt struct {
ID string
ChallengeID string
Number int
NotBefore time.Time
}
type Sender interface {
Send(ctx context.Context, idempotencyKey string) error
}
func Dispatch(ctx context.Context, now time.Time, a Attempt, s Sender) error {
if a.Number < 1 || a.Number > 3 || now.Before(a.NotBefore) {
return errors.New("attempt outside retry policy")
}
err := s.Send(ctx, a.ID)
if errors.Is(err, context.DeadlineExceeded) {
return ErrUnknownOutcome
}
return err
}
Three attempts in this example are a policy choice, not a universal safety number. Pick the cap from the authentication threat model and the delivery SLO, then load-test the resulting peak. If 8,000 sellers can request a code in a five-minute class-release window and every request is allowed three transport attempts, the queue must be analyzed against 24,000 potential sends, not the 8,000 successful-path requests shown on a product chart. That is the capacity-planning reflex that prevents a retry policy from becoming its own incident.
Keep code validity separate from delivery attempts. A transport retry can carry the still-valid challenge under the system's policy; a user-requested replacement should invalidate the prior challenge atomically. The verification store, rather than an SMS callback, remains authoritative for whether a code may be accepted.
Error normalization also deserves skepticism. Preserve the downstream raw status in restricted diagnostic storage, but map it to stable internal classes such as policy_denied, temporary_transport, permanent_destination, delivered, and unknown. Alerting and retry logic should consume those internal classes. Otherwise, a wording or taxonomy change at the transport boundary can alter production behavior without a deliberate application release.
Build, buy, or split the control plane?
The hard part is not opening a network connection. It is maintaining route eligibility, reconciling asynchronous outcomes, controlling abuse, and furnishing evidence during a review. A platform team should price those obligations in engineer time and on-call exposure rather than reducing the decision to per-message cost.
| Ownership model | Team owns | External dependency owns | Main operational risk |
|---|---|---|---|
| Managed transport | challenge policy, evidence index, normalization, SLOs | carrier connectivity and transport events | lock-in through proprietary status and identity models |
| Self-hosted gateway | all controls plus gateway capacity and routing | underlying network interconnects | larger on-call surface and policy maintenance load |
| Split control plane | policy, audit trail, portable attempt model | channel-specific transport | boundary drift between internal decisions and external outcomes |
For most teams, the split is the useful design target even if one managed transport handles all production traffic today. Keep challenge state, eligibility decisions, idempotency keys, and normalized outcomes in the application boundary. Put channel submission behind a narrow interface. This does not promise painless portability; sender approvals and delivery semantics are not interchangeable. It does prevent the authentication workflow from becoming a mirror of one provider's webhook vocabulary.
The main limitation of this approach is staffing. It is not suitable for a small team that cannot cover policy-data reviews, event reconciliation, and an authentication pager; in that case, outsourcing more of the transport and sender-policy operation can reduce operational load, provided the team still retains challenge security and an exportable evidence trail. At the other extreme, an organization with unusual routing controls or mandatory in-house custody may accept the larger self-hosted on-call surface. The trade-off is explicit: the split model buys control and portability by adding another consistency boundary, so a stale policy cache can deny valid traffic and a broken event mapper can leave outcomes unknown. Those failure modes need owners before the architecture earns production traffic.
Cost belongs in the review, but it is not the verdict. Compare the staffed operating model, evidence-retention burden, regional coverage required by the business, failure isolation, and the time needed to approve a sender-policy change. A cheaper send that expands the pager surface or cannot produce the required audit trail is not obviously cheaper.
Verify the fix before reopening traffic
Start with replayable synthetic events that contain no real phone number or patient data. Exercise admission failure, policy denial, transport timeout, delayed success, duplicate terminal callbacks, expired challenge, and a fallback-channel decision. Each case should yield one final challenge state and an append-only sequence of attempts. Callback handlers must tolerate duplicates and out-of-order arrival.
Then canary a single policy version and watch ratios, not anecdotes: accepted challenges that produce a dispatch attempt, attempts by normalized outcome, time to terminal outcome, successful verification before expiry, resend requests per challenge, and fallback activation. Segment by route profile and region, while keeping labels bounded; phone numbers and attempt IDs do not belong in metric dimensions.
Define the SLO at the user boundary. “99% of accepted transport requests returned success” says little about a seller entering a code. A better service indicator measures the proportion of eligible challenges that reach successful verification within the approved time window, with policy-denied requests reported separately because they represent control-plane readiness rather than transport latency.
Rollback has two levers. First, disable the newly deployed policy profile so no fresh attempts select it. Second, drain or quarantine queued attempts tied to that version; do not blindly replay them through the previous sender identity, because the old identity may not be eligible for the same region or message class. Existing valid challenges can be offered a separately approved fallback path.
Keep the evidence compact and useful: configuration revision, reviewer, evidence reference, eligibility decision, attempt transitions, normalized outcome, and retention class. Restrict access and avoid message secrets. This record should answer who authorized the route, what the system decided, and why a retry did or did not occur.
Done means more than seeing a code arrive on one handset. The canary meets its verification SLO, policy denials are explainable, unknown outcomes stay below the error budget, the queue survives the modeled resend ceiling, and rollback has been rehearsed. Only then should the route carry the full seller-order workload or a telehealth login flow.
References
- OWASP, “Forgot Password Cheat Sheet”: https://cheatsheetseries.owasp.org/cheatsheets/Forgot_Password_Cheat_Sheet.html
- Yahoo, “Sender Best Practices”: https://senders.yahooinc.com/best-practices/
Top comments (0)