YorkHolloway3257

Posted on Oct 30

Paging Signals for Blocked Marketplace Numbers in SMS OTP 2FA Suppression Lists

#sms #sre #security

The page for a Node.js SMS OTP 2FA auth flow says seller_login_otp_delivery_rate_low. The on-call sees 1,842 attempts, a falling delivery ratio, and no answer to the only useful first question: did eligible recipients fail, or did the application ignore its suppression list and keep trying numbers that had already been blocked?

TL;DR: put a durable recipient-eligibility check before the SMS send, return the same public response for eligible and suppressed numbers, and record an internal reason code without creating an OTP for a blocked destination. Then alert on failures among eligible sends, not on the undifferentiated request count. For a marketplace seller opening a new-order notification, that separation prevents known opt-outs, permanent delivery failures, and abuse blocks from turning into provider traffic, misleading conversion data, and a noisy page.

This is an integration-effort decision disguised as a messaging feature. The small implementation is a synchronous policy boundary, an OTP store, and an adapter with one narrow Send method. The hard work is making every path preserve account-enumeration resistance while leaving enough evidence to explain the page at 3 a.m.

How should an SMS OTP 2FA suppression list block a number?

Work backward from the page. A seller receives a new-order email, follows the link, and requests a login code. The authentication service normalizes the destination, checks whether a code may be sent, creates a short-lived challenge only when it may, and submits the message. A delivery callback later changes the attempt's state. Those are different events; treating accepted as delivered makes the dashboard comforting and the incident opaque.

The earlier signal is a change in the eligibility mix. If the proportion of requests rejected by local policy jumps, delivery infrastructure may be healthy. The cause could be repeated attempts against one suppressed number, a bad import, or an abuse campaign. None should page the carrier-delivery responder. A separate security or data-quality alert can own that condition.

That distinction is the incident.

If eligible submissions rise while callbacks stop progressing, the delivery path deserves attention. If the send adapter reports timeouts or transient failures, the integration boundary is failing. The page must say which population moved and which state transition stalled.

One graph cannot do that.

Put suppression before challenge creation

The suppression record needs a normalized address, a reason, its source, and timestamps. Reasons should be operationally distinct: an explicit opt-out is not the same fact as a permanent delivery failure, an administrator block, or a temporary abuse hold. The decision can still collapse them to deny, but operators need the original category for remediation and audit.

Normalization belongs at the boundary. Store phone numbers in E.164 form only after parsing and validating them for the intended calling context; do not invent normalization by stripping punctuation and prepending a country code. A malformed or ambiguous destination is ineligible, and the external response remains indistinguishable from the accepted path.

The architecture is the same in a Node.js transactional auth service, although the example is in Go to make the interfaces and error paths explicit. It deliberately does not expose whether a seller exists or why a number was denied. It also avoids creating a challenge on the suppressed path, which keeps verification state aligned with messages the system actually attempted to send.

package otp

import (
    "context"
    "crypto/rand"
    "crypto/sha256"
    "encoding/hex"
    "errors"
    "time"
)

var ErrTryLater = errors.New("request cannot be processed")

type Eligibility struct {
    Allowed bool
    Reason  string
}

type Suppressions interface {
    Check(ctx context.Context, e164 string) (Eligibility, error)
}

type Challenges interface {
    Create(ctx context.Context, sellerID, digest string, expiresAt time.Time) error
}

type Sender interface {
    Send(ctx context.Context, e164, body, idempotencyKey string) error
}

type Audit interface {
    Record(ctx context.Context, event string, fields map[string]string)
}

type Service struct {
    suppressions Suppressions
    challenges   Challenges
    sender       Sender
    audit        Audit
    now          func() time.Time
}

func (s *Service) Request(ctx context.Context, sellerID, e164 string) error {
    decision, err := s.suppressions.Check(ctx, e164)
    if err != nil {
        return ErrTryLater // Fail closed when recipient policy is unavailable.
    }
    if !decision.Allowed {
        s.audit.Record(ctx, "otp_suppressed", map[string]string{
            "reason": decision.Reason,
        })
        return nil // The HTTP layer returns the same generic response as success.
    }

    code, digest, err := newCode()
    if err != nil {
        return ErrTryLater
    }
    idempotencyKey := sellerID + ":" + digest
    if err := s.challenges.Create(ctx, sellerID, digest, s.now().Add(5*time.Minute)); err != nil {
        return ErrTryLater
    }
    if err := s.sender.Send(ctx, e164, "Your seller login code is "+code, idempotencyKey); err != nil {
        return ErrTryLater
    }
    return nil
}

func newCode() (string, string, error) {
    var raw [4]byte
    if _, err := rand.Read(raw[:]); err != nil {
        return "", "", err
    }
    n := uint32(raw[0])<<24 | uint32(raw[1])<<16 | uint32(raw[2])<<8 | uint32(raw[3])
    code := []byte{
        byte('0' + (n/100000)%10), byte('0' + (n/10000)%10),
        byte('0' + (n/1000)%10), byte('0' + (n/100)%10),
        byte('0' + (n/10)%10), byte('0' + n%10),
    }
    sum := sha256.Sum256(code)
    return string(code), hex.EncodeToString(sum[:]), nil
}

This sample omits the HTTP handler, phone-number parser, encryption of recipient data, rate limiter, and verification transaction because pretending those fit into a compact snippet would be dangerous. The six-digit code is generated with crypto/rand; modulo conversion introduces a tiny distribution bias, so a production implementation that requires perfectly uniform sampling should use rejection sampling. The digest also needs a server-held secret, such as an HMAC key, because a bare hash of a six-digit value is cheap to enumerate. Those details belong in the challenge implementation, along with single use, a short expiration, an attempt limit, and invalidation when a replacement is issued.

OWASP's forgot-password guidance maps cleanly to this login flow: respond consistently for existing and nonexistent accounts, keep response timing consistent, rate-limit requests, use a cryptographically secure random source, store codes securely, make them single use, and expire them. SMS is possession evidence, not proof that the rightful person still controls the account under every threat model. NIST SP 800-63B treats use of the public switched telephone network as a restricted authenticator and requires considering risks such as SIM changes and number porting. High-risk seller actions need a stronger factor or a step-up path.

Instrument the decision, not the phone number

The instrumentation change is small but opinionated. Emit a counter at each decision boundary with bounded labels: allowed, opt_out, permanent_failure, admin_block, abuse_hold, policy_error, submit_error, and callback outcomes defined by the delivery integration. Never put a phone number, seller ID, order ID, message ID, or raw provider error in a metric label. High-cardinality identifiers belong in access-controlled logs or traces with retention rules.

A useful trace links the login request to the eligibility decision, challenge record, submission, and callback through internal correlation IDs. The order ID does not belong in the SMS or telemetry merely because the flow began at a new-order notification. Keep the authentication message sparse: purpose, code, expiration context, and no clickable link. That reduces data exposure and makes phishing variants less persuasive.

I distrust a delivery-rate panel unless its denominator is stated. Define at least these populations separately:

Signal	Denominator	What it answers
Suppression ratio	valid, normalized requests	Did recipient eligibility change?
Submission failure ratio	eligible send attempts	Is the synchronous integration failing?
Confirmed delivery ratio	accepted submissions with a terminal callback	Is downstream delivery completing?
Verification success ratio	created challenges	Can sellers finish authentication?

The last metric catches a different class of incident: messages can be delivered while codes are rejected because clocks, challenge replacement, hashing, or attempt accounting is wrong. It should not be folded into carrier delivery.

Test the transitions that dashboards hide

Unit tests should exercise policy errors, every suppression reason, duplicate requests, challenge-store failure, send failure, expiration, attempt exhaustion, and one-time consumption. Integration tests should prove that denied requests create neither a challenge nor a send call, while the public status and response shape match an allowed request. Measure response-time distributions for both paths; adding a fixed sleep is usually a brittle substitute for designing equivalent work and applying a minimum response envelope.

Callback tests need replay, reordering, an unknown message ID, and a transition from an intermediate state to each documented terminal state. Make updates idempotent. A late callback must not resurrect an expired challenge or change authentication success; delivery evidence and challenge validity are separate state machines.

Deployment deserves the same skepticism. Backfill and validate the suppression store before enforcing it, run the decision in observe-only mode, compare its result with the existing path, then enable denial for one deterministic slice. Rollback must disable enforcement without deleting suppression facts. Otherwise a hurried rollback turns a delivery incident into an opt-out violation.

That trade-off is intentional: observe-only deployment takes longer, but it exposes normalization and import mistakes before they can block a seller's legitimate login.

Provider emulators are useful for adapter contracts, but they cannot establish real handset delivery behavior. Keep a small, controlled end-to-end test set, do not use real seller numbers, and ensure test destinations cannot receive production order data. The acceptance test is a state transition with evidence, not a green response from Send.

Set pages around actions, then count their cost

Page only when a human has an immediate action. A sustained rise in eligible submission errors can trigger adapter failover or traffic reduction. Missing callbacks can trigger reconciliation and escalation. A suppression spike should normally create a ticket or security signal unless it threatens the policy service itself. The alert payload should include the affected transition, denominator, start time, deployment correlation, and a runbook query that does not reveal recipient data.

Thresholds need a minimum event count as well as a ratio. With two eligible sends, one failure is 50%; waking someone for that is arithmetic theater. A longer window reduces flapping but delays detection during a marketplace sales peak, while a shorter window catches sharp regressions and pays in false positives. Choose the window from traffic volume and the time required to mitigate, then validate it against historical, labeled events rather than selecting a round percentage because it looks tidy.

There is a real false-positive cost. Every unactionable page trains the responder to scan the graph, acknowledge, and return to sleep; meanwhile the signal that should have fired earlier was the policy-decision shift, quietly mixed into a denominator it did not belong in. The design is complete when a responder can name the failed transition from the page, and when suppressed recipients never enter the delivery pipeline at all.

DEV Community