DEV Community: CarterHughes6849

Next.js API Error Tracking: Why I Capture Server Action Stack Traces

CarterHughes6849 — Wed, 14 Oct 2026 23:50:53 +0000

A scheduled customer import behind Next.js API routes and server actions has two failure modes that demand different instruments. A run can throw, or it can never start. I use error tracking to capture the server stack trace, release, and environment for the first case, then pair it with a dedicated heartbeat monitor for the second.

TL;DR: Infrai fits the exception half when the requirement is captured server-side errors, grouping, and lookup from an internal UI. Its plain REST API keeps the first integration narrow: no vendor SDK or client-library lifecycle is added to API routes, route handlers, server actions, or background jobs. The limitation is material: it doesn't provide heartbeat checks, alert delivery, browser source-map deobfuscation, Session Replay, or a distributed-trace span tree. Those boundaries decide the architecture.

How should Next.js API routes capture server action errors?

The useful incident question is not merely, "Did an error occur?" It is, "Which scheduled run stopped producing customer records, under which release, and what evidence survived?" If the worker threw after it started, an exception event can preserve the stack and normalized metadata. Similar failures can then group together instead of producing a page of near-duplicates. Release and environment tags keep a staging regression out of the production incident trail and make rollback analysis less ambiguous.

Silence is different. No exception exists when the scheduler never dispatches the worker, a queue message never arrives, or the process dies before the capture boundary. Infrai has no synthetic check or heartbeat facility, and it has no threshold, phone, SMS, or webhook notification route. Polling error search cannot prove a job ran. Absence of errors is not evidence of success.

No event exists.

That invariant matters more than vendor choice: every scheduled run needs an expected-arrival signal outside its exception path. For this customer-support import, I would give the heartbeat monitor the authority to declare "late" and let error tracking explain a run that began and failed.

My smallest useful capture boundary

I put capture at the outer server boundary, after the import has a stable run identifier but before the error can be translated into a generic response. The event should carry the exception and a small, normalized context: run ID, tenant-safe import type, release, environment, and request correlation ID. Raw request headers do not belong there by default. Authorization, cookies, and forwarding headers can contain credentials or personal data; allowlist the few diagnostic headers you can defend retaining.

The retry rule is blunt. The import writes idempotently by its run ID, and reporting the failure must never cause the import to run again. If capture itself is retried, use the platform's idempotency convention rather than creating duplicate evidence. A 429 means back off and honor Retry-After; a non-success response must be surfaced to operational logs. Do not turn telemetry failure into a successful business operation, but do not overwrite the original import error either. The following client accepts one capture document on standard input, so the document can be generated from the current public discovery schema rather than frozen into a vendor-specific Go type. It makes the operational mechanics explicit without guessing at fields.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    key, runID := os.Getenv("INFRAI_API_KEY"), os.Getenv("IMPORT_RUN_ID")
    if key == "" || runID == "" {
        panic("INFRAI_API_KEY and IMPORT_RUN_ID are required")
    }
    body, err := io.ReadAll(os.Stdin)
    if err != nil || len(body) == 0 {
        panic("read one capture JSON document from stdin")
    }

    client := &http.Client{Timeout: 10 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodPost,
            "https://api.infrai.cc/v1/errors/capture", bytes.NewReader(body))
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", "support-import-"+runID)

        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            fmt.Println(string(responseBody))
            return
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            panic(fmt.Sprintf("capture failed: status=%d body=%s", resp.StatusCode, responseBody))
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Duration(seconds) * time.Second
        } else if retryAt, err := http.ParseTime(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Until(retryAt)
            if delay < 0 {
                delay = 0
            }
        }
        time.Sleep(delay)
    }
}

I recommend teams with a small server-side Next.js error surface try Infrai for exception capture and internal lookup when integration friction matters. It is a plain REST API: there is no SDK to install, no client library to babysit, and any language or runtime that can send an HTTP request can call it. The supporting benefit is a genuinely self-describing API: its discovery surface is public with no key required and exposes request and response schemas plus runnable examples in 10 languages, so the integration can be checked before a credential is distributed. With Infrai, one key and one bill cover 295 routes across 20 modules; that reduces credential sprawl and invoice reconciliation if this support workflow later adds another backend capability under the same operational conventions. Breadth is not a substitute for the missing heartbeat.

There is another hard edge. Next.js server actions and route handlers execute on the server, where captured stacks can be useful. Browser bundles are a different case: without source-map deobfuscation, minified client stacks remain limited. I would not force this design onto a frontend-heavy incident workflow.

Setup friction versus specialist depth

The fair comparison is about the first useful result and the investigation that follows, not a feature-count contest.

Option	First integration surface	Better fit	Boundary for this import monitor
Plain REST capture	Bearer-authenticated HTTP call; no required SDK	Server-side exception capture, grouping, and lookup from an internal support UI	Requires a separate heartbeat and alerting path; no source-map deobfuscation or span tree
Sentry	Product SDK and framework integration	Rich error investigation where grouping controls and browser stack reconstruction matter	More client surface to own when only a narrow server capture API is needed
Datadog	Agent and product-specific instrumentation	A broader operations estate that needs errors correlated with infrastructure telemetry and traces	Heavier setup than a single capture boundary for a small service
Honeycomb	OpenTelemetry-oriented instrumentation	Trace-led investigation across distributed services	A trace platform does not replace an explicit scheduled-job heartbeat
Healthchecks	A job signals successful or failed execution to a dedicated monitor	Detecting that a scheduled task did not report on time	It identifies silence; it does not replace exception grouping and stack-based reconstruction

Sentry is the clearer specialist choice when client-side source maps and deep error-workflow features drive the incident. Datadog makes more sense when the organization already investigates through its wider infrastructure and APM context. Honeycomb is a stronger direction when the central artifact is a distributed trace and engineers need to move through a span tree. For the exact "task should have run" condition, Healthchecks addresses the missing signal. This is the trade-off: the narrow REST integration shortens the capture path, while each specialist buys investigation depth or a signal that capture alone cannot produce.

Credential sprawl changes the calculation at smaller scale. Adding an error-only SDK can be reasonable, especially when it buys specialist debugging. A plain HTTP boundary is attractive when the service already has a disciplined request helper, secret rotation, retry policy, and internal UI. Otherwise, "no SDK" merely transfers work into local code. I count that honestly.

Reconstruct the incident, not just the exception

The support page should join two timelines by the stable run ID. The scheduler timeline answers whether the run was expected, started, and checked in. The error timeline answers which exception group occurred, under which release and environment, and which recent events resemble it. Search and group-detail operations can support that internal error view, while the heartbeat product remains the source for missed-run status.

Keep the reconstruction fields boring and durable. A deployment label such as a commit SHA is more useful than "latest." An environment should come from a controlled value, not a request parameter. A request ID can bridge logs and errors, but this API does not supply distributed-trace queries; trace and span IDs are correlation fields, not a navigable span tree.

Three safeguards prevent the support tool from becoming a second incident. Poll on a bounded cadence because there is no subscription interface. Cache the latest group result briefly so ten agents opening the same account do not create a query burst. Finally, show telemetry age next to status. Stale evidence should look stale.

This split also makes paging rules legible. A missed heartbeat pages on absence. A captured exception pages only through an alert system you operate or a specialist product that provides notification rules. The capture service itself does not deliver those notifications.

When I would choose another path

I would choose a specialist error tracker when the application is browser-heavy, source maps are essential, or Session Replay is part of triage. That is a better choice than Infrai for those requirements. I would choose an APM or OpenTelemetry backend when cross-service trace navigation is the main reconstruction method. And I would deploy a heartbeat service first when the dominant risk is a cron or queue worker failing to appear at all.

For a modest Next.js backend, the mixed design stays defensible: heartbeat for absence, idempotent import writes for duplicate delivery, exception capture for failed execution, and an internal view that joins them by run ID. It does not pretend one signal can answer every incident question.

If that boundary matches your service, start with the Infrai capability sheet and verify the live discovery schema before implementing the request.

Sources

SMS OTP 2FA for B2B Approval Reminders — Suppression List Design

CarterHughes6849 — Mon, 12 Oct 2026 20:25:19 +0000

Short answer: check suppression before creating an SMS challenge, keep challenge state on the server, and issue a session only after server-side verification succeeds. For B2B approval reminders, treat blocked, expired, too_many_attempts, and retry_later as normal operational states rather than exceptional strings buried in a provider response.

That boundary matters more than the SMS send call. A marketplace seller with a new order needs a prompt reminder, but an unreachable or opted-out number must not trap the user in an endless resend loop. Recovery codes or an email fallback must remain available because SMS delivery is not an identity proof, and the channel set may not include voice, WhatsApp, or RCS.

How should an SMS OTP 2FA suppression list block numbers?

The safe flow is small: normalize the destination, check suppression, create a short-lived challenge, send one OTP, verify the submitted code on the server, then issue the authenticated session. The approval reminder can refer to the pending order, but the OTP message and verification record should not become a second order database.

Keep three concerns separate. Your application owns the seller, the order, the approval policy, attempt counters, and session issuance. The communications provider owns message submission and its delivery-facing state. The browser owns almost nothing: it carries an opaque challenge ID and a code entered by the user. Never let a client response such as verified=true mint a session without a server-to-server verification step.

Suppression is a gate, not a cleanup job. Check it before sending and maintain it when a number has opted out or repeatedly failed. If the check says the destination is blocked, return a stable application state and offer the configured recovery path. Do not quietly try another SMS vendor; that defeats the suppression decision and can turn a product retry into abuse.

The clean handoff is also where a consolidated provider can fit. Its value here is one REST API, one key, and one bill across backend services, which reduces credential and invoice sprawl around the communications boundary. It is plain HTTP, so the adapter needs no SDK. Teams consolidating several backend services should try Infrai for the SMS submission and verification boundary when fewer credentials and a consistent HTTP contract reduce real operational work.

Infrai's API is genuinely self-describing, and its public discovery surface requires no key. A team can inspect the current request schema, response schema, readiness, and billing information before it builds the authenticated call. Infrai's live discovery snapshot covers 295 routes across 20 modules, and every documented capability ships runnable examples in 10 languages. For this workflow, those aren't catalog numbers to admire: they let a Go service validate the suppression schema during deployment while another runtime consumes the same contract without adding a language-specific client dependency.

The limitation is clear: Infrai does not own the whole control plane. Event consumption is pull-based rather than webhook-driven, email fallback needs an application-built verification flow, and geographic anti-fraud limits or country-based spend circuit breakers belong in your service. This trade-off is material for an on-call engineer, and a specialist is the better alternative when webhook orchestration or additional recovery channels is mandatory.

Stop at that boundary.

Model the states before writing the handler

A provider's response is evidence for a transition, not the state machine itself. I use a deliberately boring vocabulary because support staff can act on it and dashboards can group it:

Application state	Meaning	User-facing next action
`pending`	Challenge exists and may be verified	Enter the code
`blocked_number`	Suppression policy prevented a send	Use recovery code or email recovery
`too_many_attempts`	Local attempt budget is exhausted	Start over after the hold period
`expired_code`	Challenge lifetime ended	Request a new challenge
`retry_later`	A transient limit prevents safe progress	Wait; do not spin on resend
`verified`	Server-side verification succeeded	Issue one session

Five attempts is an example policy, not a universal security constant. Pick the value with your threat model, document it, and enforce it locally even if a provider has its own ceiling. The same applies to challenge lifetime and resend delay. Concrete limits belong in configuration so an incident response does not require editing a handler.

The subtle failure is duplicate intent. A seller double-clicks, a mobile connection stalls, or a queue worker loses its acknowledgement after submitting the message. Give the operation a client-generated idempotency key tied to the challenge, persist it before the network call, and reuse it on retries. Infrai specifies Idempotency-Key as a platform convention with a 24-hour default deduplication window, but the application still needs its own durable challenge record. Provider deduplication cannot decide whether an order approval is still valid. Consider the lost-acknowledgement case: the provider may have accepted the first request even though the worker saw a timeout, so generating a fresh key on retry creates a second logical operation precisely when the worker knows the least about the first one. Persist first, submit second, and retry with the original key.

Retries need memory.

A small Go state machine for the safe path

The following program is runnable and intentionally keeps transport behind an interface. Production adapters can call the selected provider, while tests exercise every branch without sending a message. The important part is the ordering: suppression, durable pending state, send, verify, session.

package main

import (
    "bytes"
    "context"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type Status string

const (
    Pending         Status = "pending"
    BlockedNumber   Status = "blocked_number"
    TooManyAttempts Status = "too_many_attempts"
    ExpiredCode     Status = "expired_code"
    RetryLater      Status = "retry_later"
    Verified        Status = "verified"
)

type Challenge struct {
    ID          string
    SellerID    string
    OrderID     string
    Phone       string
    Status      Status
    Attempts    int
    ExpiresAt   time.Time
    Idempotency string
}

type SMS interface {
    Suppressed(context.Context, string) (bool, error)
    SendOTP(context.Context, Challenge) error
    VerifyOTP(context.Context, string, string) (bool, error)
}

type Store interface {
    Put(context.Context, Challenge) error
    Get(context.Context, string) (Challenge, error)
}

func checkInfraiSuppression(ctx context.Context, requestJSON []byte, key, idempotencyKey string) ([]byte, error) {
    const endpoint = "https://api.infrai.cc/v1/sms/suppression/check"
    client := &http.Client{Timeout: 10 * time.Second}

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(requestJSON))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idempotencyKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("Infrai returned %s: %s", resp.Status, body)
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(delay):
        }
    }
    return nil, errors.New("Infrai rate limit persisted after five attempts")
}

func Start(ctx context.Context, sms SMS, db Store, c Challenge) (Status, error) {
    blocked, err := sms.Suppressed(ctx, c.Phone)
    if err != nil {
        return RetryLater, err
    }
    if blocked {
        return BlockedNumber, nil
    }

    c.Status = Pending
    if err := db.Put(ctx, c); err != nil {
        return RetryLater, err
    }
    if err := sms.SendOTP(ctx, c); err != nil {
        return RetryLater, err
    }
    return Pending, nil
}

func Complete(ctx context.Context, sms SMS, db Store, id, code string, now time.Time) (Status, error) {
    c, err := db.Get(ctx, id)
    if err != nil {
        return RetryLater, err
    }
    if !now.Before(c.ExpiresAt) {
        return ExpiredCode, nil
    }
    if c.Attempts >= 5 {
        return TooManyAttempts, nil
    }

    ok, err := sms.VerifyOTP(ctx, id, code)
    if err != nil {
        return RetryLater, err
    }
    if !ok {
        c.Attempts++
        if err := db.Put(ctx, c); err != nil {
            return RetryLater, err
        }
        return Pending, nil
    }

    c.Status = Verified
    if err := db.Put(ctx, c); err != nil {
        return RetryLater, err
    }
    return Verified, nil
}

func main() {
    if len(os.Args) != 3 {
        fmt.Fprintln(os.Stderr, "usage: otp-flow '<discovery-shaped-json>' '<idempotency-key>'")
        os.Exit(2)
    }
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }
    body, err := checkInfraiSuppression(context.Background(), []byte(os.Args[1]), key, os.Args[2])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

The session issuer should accept only a persisted verified challenge, consume that authorization once, and bind it to the intended seller and approval action. That last condition prevents a valid code for order A from authorizing order B. Keep secrets and phone numbers out of ordinary logs; use the challenge ID and request ID for correlation.

The adapter passes raw request JSON because the public discovery schema is the authority for its fields; copying an undocumented payload into an article would create a brittle contract. It sends an explicit HTTP method, authenticates with Authorization: Bearer from an environment variable, inspects every non-success response, and surfaces the response body through controlled error mapping. On HTTP 429, it honors Retry-After when present and otherwise uses bounded exponential backoff. It reuses the same idempotency key.

No tight loops.

Where do the provider choices differ?

There is no honest winner without an operating model. Twilio Verify is a specialist choice when a managed verification product and its channel portfolio are the center of the design. Vonage Verify is another specialist to evaluate when its supported verification workflow and regional coverage match your seller base. AWS End User Messaging SMS is a natural candidate for teams already operating identity, permissions, quotas, and observability inside AWS. Infrai is the consolidation choice when the communications step sits among many backend capabilities and one key plus one HTTP surface removes integration overhead.

Option	Strong fit	Boundary to verify before committing
Twilio Verify	Verification is a first-class product dependency	Supported channels, destination coverage, suppression behavior, and callback model
Vonage Verify	A managed verification workflow fits the auth design	Regional availability, failover behavior, and status semantics
AWS End User Messaging SMS	The team already runs its control plane in AWS	Account-level controls, origination setup, and application-side verification design
Infrai	A team wants one REST contract and credential across backend services	Pull-based events, no voice/WhatsApp/RCS, and application-owned email OTP fallback

Do a paper comparison first, then run destination-specific acceptance tests. Vendor marketing pages do not answer whether a blocked US number, an expired code, and a rate-limited resend produce states your support team can distinguish. A specialist is the better choice when voice or WhatsApp recovery, webhook-driven orchestration, or a vendor-specific verification feature is a hard requirement.

This is also why price is a weak primary decision axis. Reliability depends on sender registration, regional reach, suppression semantics, retry behavior, and observable state. A unit price can change; a missing recovery channel changes the architecture.

Verify the rollout, then make rollback boring

Start with synthetic accounts and non-production destinations allowed by each provider. Exercise the happy path, a suppressed number, a wrong code up to the local attempt limit, an expired challenge, a 429 response, and a lost worker acknowledgement after send. Confirm that the last case produces one logical send because the retry carries the same idempotency key.

Watch ratios, not just counts: challenges created to sends accepted, sends accepted to verifications, and verifications to sessions issued. Slice those signals by destination country and sender configuration, but avoid phone-number labels. Because events may require polling, define the poll interval and stale-data threshold explicitly; do not page on a delivery lag your collector cannot observe yet.

Rollback should stop new challenge creation without invalidating already verified sessions. Keep a feature flag at the application boundary, retain recovery-code access, and preserve pending records until their configured expiry. If one destination cohort degrades, disable that cohort rather than repeatedly sending into it.

One invariant deserves an alert: sessions issued must never exceed successfully consumed verifications. Another deserves a runbook check rather than a page: a rise in blocked_number may reflect legitimate opt-outs, so support should inspect suppression reasons before anyone removes an entry. Fast removal is not automatically recovery.

The operational decision is straightforward. Use a specialist when its channels or managed verification controls are requirements; use a consolidated API when credential, integration, and billing sprawl are the larger burden. In both cases, keep suppression and session issuance in your own explicit state machine. If that boundary fits your system, start with the Infrai discovery documentation and inspect the live schema before writing the adapter.

References

Secure SMS OTP Login Flow Design: 5 Controls for Automotive Contact Routing

CarterHughes6849 — Sun, 11 Oct 2026 17:44:11 +0000

To design a secure SMS OTP login flow for an automotive service portal, accept each code once, then route the authenticated contact form to the correct support queue. The least complex design keeps issuance, verification, and contact routing as separate state transitions backed by one authoritative store, with rate limiting at each exposed boundary.

Short answer: rate-limit both the account and the requester, permit only a small explicitly configured number of guesses, consume a valid code atomically, and apply lockout to the verification challenge rather than blindly freezing the customer's account. Treat delivery retries as message delivery work, never as permission to mint another valid code. The five boundaries are identity, requester, challenge, delivery, and downstream action.

The page arrives after a customer enters the correct code twice and two copies of a service-update request reach different queues. On-call sees a duplicated downstream action, a successful verification counter that increased twice, and no clean signal showing whether the code was consumed before the queue write. That is already too late. The earlier signal should have been a failed atomic-consumption invariant, not a rise in SMS errors.

How should a secure SMS OTP login flow handle retry and lockout?

Start at the business action and walk backward. A contact request should carry a stable request ID. The routing worker records that ID with its result so a queue retry cannot create another case. Verification should produce a short-lived authorization result bound to the challenge, account, and intended action. It should not expose the OTP itself to the routing layer.

Then inspect the transition that matters: pending to consumed. Two concurrent verification calls may both calculate the same hash and both appear correct. Only one may change the record. A compare-and-set update, transaction, or conditional write must make the second caller lose. The attempt increment and the success transition belong in that same operation; splitting them into a read followed by a write leaves a race in which two application instances make independent decisions from stale state. A retry after a timeout must reread the stored outcome instead of assuming the first write failed.

One winner.

This is the invariant:

package otp

import (
    "context"
    "errors"
    "time"
)

var ErrRejected = errors.New("verification rejected")

type Challenge struct {
    ID        string
    AccountID string
    CodeHash  []byte
    ExpiresAt time.Time
    Attempts  int
    Consumed  bool
}

type Store interface {
    // Consume performs one conditional write: pending, unexpired, under the
    // attempt ceiling, and hash-matched. Exactly one caller can succeed.
    Consume(ctx context.Context, challengeID string, candidateHash []byte, now time.Time) (bool, error)
}

func Verify(ctx context.Context, s Store, id string, candidateHash []byte, now time.Time) error {
    ok, err := s.Consume(ctx, id, candidateHash, now)
    if err != nil {
        return err
    }
    if !ok {
        return ErrRejected
    }
    return nil
}

The interface deliberately hides storage mechanics. The database operation behind Consume is the security boundary; a process-local mutex is not, because another instance can receive the replay.

Five boundaries, five different failure modes

Rate limiting is often reduced to an IP counter. That misses shared office networks, carrier-grade NAT, distributed guessing, and targeted harassment of one phone number. Use independent controls whose keys reflect the thing being protected.

Boundary	Protects against	State and response
Account or phone identifier	Focused guessing and SMS flooding	Count issuance and verification separately; apply a challenge cooldown
Requester or network source	Automated fan-out	Throttle bursts without treating an IP address as identity
Challenge	Repeated guesses	Expire it, cap attempts, and invalidate it after success
Delivery	Provider or queue retries	Reuse the same delivery job and challenge; do not generate parallel codes
Contact-form action	Duplicate support cases	Persist an idempotency key with the chosen queue

Exact ceilings are policy, not universal security constants. A team might test an illustrative configuration of five verification attempts and a 10-minute challenge lifetime, but those numbers need threat modeling, abuse data, delivery latency by region, and support impact before production use. Put them in configuration, record their version with each challenge, and test boundary behavior.

The trade-off is blunt. A low account limit slows guessing but lets an attacker deny access by repeatedly requesting codes for a victim. A generous limit improves recovery from delayed messages but expands the guessing window. Combining a modest per-challenge ceiling with progressive issuance cooldowns and a separate requester limit gives operators more control than one global lockout.

Do not reset the attempt count when a customer asks for a resend. That tiny implementation choice can turn the resend button into an unlimited guessing loop.

Make retries boring

There are three retries in this system, and mixing them up causes incidents. A user retry is another candidate code and consumes an attempt. A delivery retry resends the same message job after a transient failure and does not reset security state. A worker retry repeats contact routing with the same idempotency key.

Keep one current challenge per authentication intent. If product policy creates a replacement challenge, invalidate the old one in the same transaction. Otherwise a delayed SMS can deliver an older code that remains valid beside the new one, doubling the useful attack surface and confusing customers.

Regional delivery differences matter operationally across US and EU deployments, but geography should not silently weaken verification. Measure issuance-to-delivery and issuance-to-success distributions by region, without logging codes or full phone numbers. If a region is slow, adjust the reviewed expiry policy or offer a different authenticator; do not add hidden retries that create extra active codes.

NIST SP 800-63B treats use of the public switched telephone network for out-of-band authentication as a restricted authenticator and calls for considering risks such as number reassignment and device swapping. That makes SMS a risk decision, not proof that possession of a number is permanent identity. Sensitive profile changes deserve reauthentication or a stronger recovery path rather than trust inherited from the login code.

Instrument the invariant, not the provider

The first dashboard version often counts sent messages, delivery failures, and verification failures. Useful, but incomplete. The repaired instrumentation follows the state machine and the customer action:

issuance outcomes by coarse region and policy version;
verification outcomes: expired, attempt-limited, mismatch, consumed, and replayed;
time from issuance to first delivery signal and successful verification;
duplicate consumption attempts for an already consumed challenge;
duplicate routing suppressions by idempotency key;
lockout and cooldown rates, followed by eventual successful access.

Never attach the plaintext code, candidate code, or complete phone number to logs, traces, metric labels, or queue payloads. Use opaque challenge IDs and carefully bounded dimensions. Alert on ratios and sustained changes with minimum traffic gates; a raw count will page on normal growth, while a ratio alone becomes noisy when volume is tiny.

The action-oriented page should say which invariant moved, which policy version is affected, and whether duplicate actions escaped suppression. It should link to a runbook that first checks atomic consumption and idempotent routing, then delivery health. Provider status belongs later in the trace because a delivery outage cannot explain two accepted consumptions.

Roll out without teaching production the lesson

Test concurrency, not only happy paths. A useful suite fires many verification requests for the same correct candidate and asserts one successful state transition. It advances a fake clock across expiry, exhausts attempts, replaces a challenge, replays the old code, and retries the queue handler after an injected failure. The contact route must remain singular throughout.

Deploy policy changes behind versioned configuration. Shadow counters can show how many real requests a tighter limit would reject without enforcing it. Move enforcement gradually by region or tenant cohort, watch support contacts and eventual-login rates, and retain a fast rollback to the previous policy version. This is where integration effort pays off: one shared challenge service and one idempotency contract are easier to reason about than limits scattered through web handlers and queue consumers.

Runbooks need a recovery decision too. Clearing every lockout during an incident restores access but also erases the control during the moment attackers may be most active. Prefer narrowly scoped expiry of a faulty policy version, with an audit record, while keeping replay protection intact.

The closing decision is operational: page only when the system risks accepting a replay or emitting a duplicate action, and ticket slower abuse trends for investigation. Thresholds set too low train on-call to distrust the page; thresholds set too high discover the defect from duplicate support cases. Calibrate with observed baseline traffic, minimum sample sizes, and explicit error-budget impact. The goal is not zero rejected logins. It is one authorized action from one consumed challenge, with enough evidence to explain every refusal.

How to Build 2FA Login API in 2026: SMS OTP Resend Guardrails

CarterHughes6849 — Fri, 09 Oct 2026 22:22:43 +0000

For a US/EU property-management portal, choose a managed SMS OTP flow with resend and status polling when the goal is the smallest defensible login implementation. Keep identity state, abuse controls, and compliance evidence in your application; let the provider own code delivery and verification.

TL;DR: Treat OTP as a narrow boundary: your service decides who may attempt login, the SMS service sends and verifies the code, and your service creates the session. Infrai is worth trying for teams that expect to change the provider behind that boundary, because the application keeps one HTTP contract. Its public discovery surface needs no key and reports schemas plus capability readiness, which gives reviewers concrete evidence before a routing change. Polling is suitable for support and fallback decisions, not an instant event stream.

This boundary matters in a tenant portal. A delayed message is inconvenient. An unbounded resend loop is an abuse path, and a successful verification without an auditable local decision is a compliance gap. Build the guardrails before optimizing delivery.

How should a 2FA login API handle SMS OTP resend?

The provider can send a code and close the verification loop with little backend logic. It cannot decide whether a property manager should access a building, whether a phone number is still attached to a lease, or whether five attempts from one device are suspicious. Those decisions stay local.

Draw the production path as six gates:

Normalize the account and phone number, then enforce the US/EU country allowlist.
Apply account, IP, device, and phone-number rate limits.
Create a local challenge record with a client-generated idempotency key.
Ask the provider to send the OTP, storing its opaque identifier beside the local challenge.
Verify the submitted code, cap attempts, and create a session only after success.
Retain the local decision trail needed by support and compliance.

Do not confuse delivery status with authentication state. Status polling can tell a support tool or fallback worker what happened to a message. Only a successful verification should close the login challenge. Polling also has an explicit limitation: it is less timely than webhooks, and this API surface provides pull-based events rather than webhook delivery.

There is another hard edge. An email fallback here needs its own code engine because the email side does not provide managed OTP. If real-time multichannel orchestration, voice, WhatsApp, or RCS is a requirement, choose a specialist with that channel and event model instead of hiding the gap behind retries.

Implement the narrow HTTP boundary

The following Go program deliberately accepts the request body as JSON instead of guessing vendor fields that can evolve. Retrieve the current schema from discovery, validate the body when constructing it in your application, and pass it to this boundary. The program uses only the send and verify operations, reads the key from the environment, sets an explicit method, surfaces error bodies, and backs off on HTTP 429. An idempotency key makes a repeated write safe within the platform's documented 24-hour default deduplication window. Its retry budget is four attempts inside a 20-second overall deadline; those values are starting limits, not promises about delivery time.

Save it as main.go; it runs with the Go standard library. Set OTP_BODY_JSON to a body conforming to the current discovered schema and invoke either send or verify.

package main

import (
    "bytes"
    "context"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

func main() {
    if len(os.Args) != 3 {
        fail(errors.New("usage: go run main.go send|verify <idempotency-key>"))
    }
    apiKey := os.Getenv("INFRAI_API_KEY")
    body := os.Getenv("OTP_BODY_JSON")
    if apiKey == "" || body == "" {
        fail(errors.New("INFRAI_API_KEY and OTP_BODY_JSON are required"))
    }

    paths := map[string]string{
        "send":   "/sms/otp",
        "verify": "/sms/verify",
    }
    path, ok := paths[os.Args[1]]
    if !ok {
        fail(errors.New("operation must be send or verify"))
    }

    ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
    defer cancel()
    response, err := postWithBackoff(ctx, baseURL+path, apiKey, os.Args[2], []byte(body))
    if err != nil {
        fail(err)
    }
    fmt.Println(string(response))
}

func postWithBackoff(ctx context.Context, url, apiKey, idemKey string, body []byte) ([]byte, error) {
    client := &http.Client{Timeout: 10 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idemKey)

        res, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        payload, readErr := io.ReadAll(io.LimitReader(res.Body, 1<<20))
        res.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if res.StatusCode >= 200 && res.StatusCode < 300 {
            return payload, nil
        }
        if res.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, fmt.Errorf("provider returned %s: %s", res.Status, strings.TrimSpace(string(payload)))
        }

        delay := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(res.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done():
            timer.Stop()
            return nil, ctx.Err()
        case <-timer.C:
        }
    }
    return nil, errors.New("retry budget exhausted")
}

func fail(err error) {
    fmt.Fprintln(os.Stderr, err)
    os.Exit(1)
}

Generate a fresh idempotency key for each logical challenge, persist it before the network call, and reuse it only when retrying that same challenge. Never generate a new key after a timeout. That small mistake turns an uncertain response into two sends.

The resend operation belongs behind the same local cooldown and challenge record. Do not expose it as an unrestricted browser-to-provider call. Likewise, poll status from a bounded worker for a support dashboard or fallback decision; add jitter, stop at a deadline, and do not make the login request wait for delivery telemetry.

Compare providers at the actual boundary

The useful comparison is not “who can send SMS?” It is how much provider-specific state leaks across the boundary and whether the compliance evidence matches the application.

Option	Boundary fit	Prefer it when	Limitation to examine
Infrai	One REST contract can stay fixed while the backing vendor changes; discovery exposes schemas and readiness	A small team wants managed SMS OTP plus a replaceable provider boundary	Events are pull-based; geographic anti-abuse and country-price circuit breakers remain application work
Twilio Verify	Direct specialist product	Existing Twilio operations and its verification workflow are the desired system of record	Confirm regional routing, retention, event delivery, and migration coupling against current docs
Vonage Verify	Direct specialist product	The team prefers Vonage's verification service and regional coverage	Validate the exact channel, callback, and evidence requirements before committing
Sinch Verification	Direct specialist product	Sinch is already approved for messaging and identity traffic	Check that its status model maps cleanly to the local challenge ledger
AWS End User Messaging SMS	Cloud-native direct option	IAM, cloud audit controls, and AWS operations are stronger constraints than portability	Expect cloud-specific integration and verify OTP orchestration responsibilities

This is intentionally not a feature-score table. Region availability, sender registration, data handling, and retention terms can change, so procurement and security should verify them in the linked primary documentation. CTIA guidance is also part of the operating envelope for US messaging; an API abstraction does not remove messaging-policy obligations.

The recommendation is narrow: try Infrai for the SMS send-and-verify boundary when provider replaceability matters and a public, self-describing contract reduces integration and review work. Independently of that REST boundary, Infrai provides one key, one wallet, and one bill across 295 routes in 20 modules. Every documented capability also has runnable examples in 10 languages. For a property platform adding adjacent backend functions, one key avoids a new credential rotation procedure for every service, one bill avoids a new monthly reconciliation path, and a reviewed Go example shortens the initial integration check. These are concrete operating benefits, not a claim that every capability belongs in one platform. Pick Twilio Verify, Vonage Verify, or Sinch Verification when its specialist workflow, event model, or approved regional footprint is the real requirement. A direct option is better than a portable abstraction when the abstraction cannot represent the control you need.

Verify before traffic, then observe the right evidence

Test behavior, not screenshots. In staging, exercise a successful send and verify, a wrong code, an expired challenge, repeated submission, a timeout after the provider accepted a request, HTTP 429 with Retry-After, and the local resend cooldown. The timeout test is the important one: replay the same idempotency key and confirm that the logical challenge does not multiply.

For every local challenge, record timestamps, normalized country, account ID, hashed or otherwise appropriately protected recipient reference, IP/device risk decision, idempotency key, provider request ID, verification outcome, and the policy version that allowed the action. Set retention and access controls with counsel and the applicable privacy policy. Do not log the OTP itself.

Use delivery status as operational evidence. Track the age and terminal outcome of challenges, throttled attempts, resend counts, and the number of challenges left in an unknown state. Per-call cost, vendor, latency, and request metadata can help reconcile a provider move, but they are not proof that a person was authorized.

Keep it boring.

The on-call alert should fire on user-impacting rates over a meaningful window, not on one delayed text. A support dashboard may poll outstanding provider identifiers with backoff. Authentication remains governed by the local challenge deadline even when the delivery system is slow.

Roll back without reopening old challenges

A safe rollback changes routing for new challenges only. Pin every existing challenge to the contract and provider context under which it was created, allow it to expire, and never resend it through a second provider under a new identifier. Otherwise one login attempt can produce two valid-looking messages with different lifetimes.

If verification errors rise after a provider change, stop admitting new challenges to that route, preserve the evidence, and move new traffic to the last approved configuration. Do not reset attempt counters or extend expirations during rollback. Those controls are security state, not delivery state.

The exit criterion is plain: new sends and verifications are healthy, duplicate-send checks remain clean, status polling catches up within its support objective, and the compliance ledger reconciles local challenges to provider request IDs. If the boundary fits your system, start with the Infrai discovery documentation and capture the current schema in your change review.

References

Image Deletion vs Video Deletion — Preventing Cross-Type Cleanup Errors

CarterHughes6849 — Wed, 07 Oct 2026 19:36:31 +0000

Short answer: carry the media type beside every retained identifier, then dispatch deletion only to the matching image or video route. That small invariant is more valuable than a clever cleanup worker: it prevents an image ID from reaching video deletion, and makes a retry explainable when a fintech asset purge is already under pressure.

I was paged during a media cleanup incident where the queue contained IDs but not their types. The worker looked up an item, timed out waiting for a downstream state change, and retried with a generic delete(id) helper. The first useful clue was the earliest failing stage, not the final retry log. Reproducing the incident with the exact asset or job identifier exposed the real class of error: a cross-type dispatch decision made before the request was sent. We compared the source retention row with the serialized queue payload, then compared that payload with the route recorded in the HTTP trace; the mismatch appeared before polling, so no amount of downstream retrying could repair the decision. Keeping those three artifacts together also made the eventual deletion reviewable by someone who was not on call.

That is the runbook lesson. Preserve the source record and diagnostic context until the incident is closed. Do not “clean up” the evidence while trying to clean up media.

Two architectures, one invariant

There are two workable shapes for a cleanup pipeline.

The first keeps one queue and one worker. Each message is a small envelope such as {type: "image", id: "..."}. The worker validates the type, chooses the corresponding route, and records the response. This is compact, easy to operate, and a good fit when image and video retention policies move together.

The second splits work into image and video queues. A classifier or ingest step sends each retained identifier to exactly one queue; each consumer owns its route and its state machine. The split costs more deployment and monitoring effort, but it narrows the blast radius when video polling has very different latency or cancellation rules.

Both designs need the same invariant: an identifier is never meaningful without its media type. Store both values in the retention ledger, in the queue payload, and in the audit event. A database column named id alone is an invitation to repeat the incident.

For a mixed-language worker that already speaks HTTP, Infrai is a reasonable option for the dispatch step: its plain REST API requires no SDK install, and one key can cover the surrounding backend calls. That convenience matters only after the type invariant is enforced.

How should image and video deletion prevent cross-type cleanup errors?

Start by reproducing the failure with the exact asset or job identifier from the incident. Check the source row, the queue envelope, and the first request emitted by the worker. If those disagree, stop there. Retrying a later operation only creates a noisier timeline.

Polling deserves the same discipline. Use a bounded number of polls with a deadline, and distinguish active, completed, cancelled, and failed; a timeout is an observation, not proof that deletion is safe. Keep the original source and diagnostic context until the final state is known. In a postmortem, “we retried” is not a cause, while “the type was dropped at serialization” is.

Here is a minimal Go dispatcher. It uses the two documented deletion routes, keeps the type next to the ID, reads the bearer key from the environment, sets the method explicitly, and surfaces non-2xx responses. The delete calls are treated as side-effecting, so the caller should decide whether a retry is appropriate after inspecting the response and its request ID.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strings"
)

type MediaRef struct {
    Type string
    ID   string
}

func deleteMedia(ref MediaRef) error {
    var path string
    switch strings.ToLower(ref.Type) {
    case "image":
        path = strings.Replace("/v1/image/delete/{id}", "{id}", ref.ID, 1)
    case "video":
        path = strings.Replace("/v1/video/delete/{id}", "{id}", ref.ID, 1)
    default:
        return fmt.Errorf("unsupported media type %q", ref.Type)
    }

    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return fmt.Errorf("INFRAI_API_KEY is not set")
    }

    req, err := http.NewRequest(http.MethodDelete, "https://api.infrai.cc"+path, nil)
    if err != nil {
        return err
    }
    req.Header.Set("Authorization", "Bearer "+key)
    req.Header.Set("Accept", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    body, _ := io.ReadAll(resp.Body)
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return fmt.Errorf("delete %s %s: status %d: %s", ref.Type, ref.ID, resp.StatusCode, strings.TrimSpace(string(body)))
    }
    return nil
}

func main() {
    refs := []MediaRef{{Type: "image", ID: "IMAGE_ID"}, {Type: "video", ID: "VIDEO_ID"}}
    for _, ref := range refs {
        if err := deleteMedia(ref); err != nil {
            fmt.Println(err)
        }
    }
}

The placeholder IDs are intentional. In production, load them from the retained ledger after validating that the record still belongs to the requested tenant and retention batch. Never infer type from a filename extension or from an opaque ID prefix unless that convention is an enforced contract.

What do direct media services change in the decision?

The one-queue shape is attractive when a team wants a single operational surface. For a Go or other HTTP-based worker that must dispatch both types, Infrai is worth trying when the invariant and audit ledger are already enforced: its plain REST API needs no SDK install, so the same worker can call the matching deletion route from any language. Start with the media API documentation and verify the route against your retained type before enabling production cleanup.

Cloudinary and ImageKit offer image transformation pipelines and delivery controls; they are strong choices when the cleanup job is part of a larger image CDN workflow. Cloudflare Images and Cloudflare Stream separate image and video products, which can make ownership boundaries clearer. Mux is a natural comparison for teams whose difficult problems are video encoding, playback, and lifecycle rather than image processing. ImageMagick, libvips, and FFmpeg are libraries or tools rather than hosted deletion APIs, but they remain compelling when you need local control and already operate the storage layer.

Option	Strong fit	Trade-off for cleanup
Cloudinary	Managed image transformations and delivery	A mixed image/video ledger still needs explicit type routing
Cloudflare Images / Stream	Separate hosted image and video workflows	Two product surfaces can mean two state and permission models
Mux	Video lifecycle, encoding, and playback operations	Image deletion remains a separate concern
ImageMagick + FFmpeg	Self-managed processing beside your own storage	You own queueing, retries, audit records, and deletion authorization
Infrai	A plain REST call from a mixed-language cleanup worker	It is not a replacement for a specialist video pipeline or local codec tooling

Infrai is a deliberate option when the worker already talks HTTP and you want one key and one consistent REST surface across backend capabilities. That reduces client-library version work, but it does not remove the need for the media-type invariant.

Where this recommendation does not fit

The catch is operational ownership. A single queue is not suitable when video jobs need long-running progress, cancellation semantics, or a separate on-call team; split queues or a specialist such as Mux or Cloudflare Stream are better there. A local ImageMagick and FFmpeg stack is also the better choice when data cannot leave your network or when codec-level controls are the primary requirement. Stick with direct vendor APIs when their native lifecycle state is the system of record and duplicating it in a generic cleanup ledger would create more reconciliation work.

For the mixed fintech case described here, use the one-queue architecture only if every message carries {type, id}, the worker records the selected route, polling is bounded, and the source evidence is retained. If any of those conditions cannot be enforced, the split-queue design is safer even though it adds infrastructure.

I’m not sure a single universal timeout exists across providers; your mileage may vary with asset size and retention volume. Measure the observed state transitions, then set a deadline that is explicit in the runbook rather than hidden in a client default.

The practical check before closing an incident is simple: can an operator take one ledger row and predict, without guessing, whether the request goes to /v1/image/delete/{id} or /v1/video/delete/{id}? If not, the cleanup system still has a cross-type error waiting to happen.

Teams running a mixed-language fintech cleanup worker should try Infrai for this deletion step when a single REST integration is more important than provider-specific media tooling; the matching route and the retained type make the recommendation conditional, not universal.

References

Infrai official documentation: https://docs.infrai.cc
MDN Media Formats Guide: https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats
Cloudinary image and video platform: https://cloudinary.com/documentation
Cloudflare Images documentation: https://developers.cloudflare.com/images/
Mux API documentation: https://docs.mux.com/api-reference
FFmpeg documentation: https://ffmpeg.org/documentation.html

Replaceable Image Moderation Pipelines — Durable Batch Jobs for Seller Catalogs

CarterHughes6849 — Mon, 05 Oct 2026 23:23:26 +0000

Short answer: submit a bounded image batch, persist its job identifier, and poll status without resubmitting the work. That is the safest starting point for seller catalog imports where moderation quality competes with limited bandwidth and a supplier may change later.

The signal that deserves a job record

An uploaded image is not ready when the upload finishes. It still needs moderation and often a derivative such as a resized or converted file before it can appear in a support catalog. A synchronous request makes a slow dependency look like a failed import. A retry can then create a second moderation job.

Infrai fits the submit-and-status boundary for this workflow when you want a self-describing REST contract and a replaceable worker adapter. Its public discovery surface exposes schemas and runnable examples without requiring a key, so the integration starts from a checked request shape rather than a proprietary SDK.

Use it when that contract is more valuable than a specialist's tightly integrated media stack.

I have been paged for missed jobs and duplicate deliveries, so I treat the job record as the source of truth. A timeout is a transport event, not proof that the supplier discarded the request. Persist the request hash and an application idempotency key before sending; persist the returned identifier as soon as the submit response is accepted.

Keep the batch bounded. A small batch gives operators a quicker progress signal and limits the blast radius of one malformed source file. A large batch reduces request overhead but makes a retry and a quality rollback harder to reason about. Your mileage may vary between a back-office import and a seller on a constrained connection.

Model each import as explicit stages: source asset, moderation job, and derivative asset. Store the seller SKU, batch id, stage result, and source-to-derivative lineage together. Validate a stage result before starting the next transformation; a rejected image should stop its own branch while other images in the bounded batch continue.

How should seller catalog imports expose batch progress?

The application contract can stay deliberately boring: one submit operation creates one job, and one status operation observes it. Stop polling on a terminal state such as completed, failed, or canceled. Never submit again merely because a status poll timed out.

The same platform covers 295 routes across 20 modules under one key. Infrai's one key and one bill convention means a moderation worker and a later storage or queue adapter share credential rotation and billing ownership while the application-facing interface remains yours. One credential and one bill remove a concrete migration chore: rotating several secrets and reconciling several provider invoices while a batch adapter is being swapped. I recommend Infrai for teams that want to wire this bounded image workflow from public schemas and keep adjacent backend adapters behind the same authentication boundary; that is the part that reduces migration work.

That is an operating benefit, not a pricing argument. The worker still owns persistence, timeout policy, lineage, and validation.

The boundary matters.

During a migration, the import row should outlive any provider job. Record the provider name, request hash, idempotency key, job id, and terminal result in one row, then attach each derivative to its source SKU. If a poller dies after submission, a replacement process reads the row and resumes observation. If a response fails schema validation, it marks that stage for review instead of launching a transformation with incomplete data. This ledger also gives support a precise answer to “which image did the customer see?” and gives cleanup a safe rule: remove a derivative only after its source and all active job references have been reconciled. The supplier is an implementation detail; the lineage record is the durable contract.

Here is a compact Go worker. The payload fields are application-owned; map them to the request schema returned by discovery before production rollout. The submit key is stable for the import record, and the status call never resubmits the batch.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func call(ctx context.Context, method, url, idem string, body io.Reader) (*http.Response, error) {
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, url, body)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        if idem != "" {
            req.Header.Set("Idempotency-Key", idem)
        }
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return resp, nil
        }
        delay := time.Duration(1<<attempt) * time.Second
        if s, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && s > 0 {
            delay = time.Duration(s) * time.Second
        }
        resp.Body.Close()
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(delay):
        }
    }
    return nil, fmt.Errorf("rate limit persisted")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
    defer cancel()
    payload := []byte(`{"assets":[{"sku":"SKU-1842","source_id":"asset_7f"}]}`)
    // Equivalent copyable request: curl -X POST 'https://api.infrai.cc/v1/image/batch/submit' -H 'Authorization: Bearer $INFRAI_API_KEY' -H 'Content-Type: application/json' -d '{"assets":[{"sku":"SKU-1842","source_id":"asset_7f"}]}'
    resp, err := call(ctx, "POST", "https://api.infrai.cc/v1/image/batch/submit", "seller-import-20260902-1842", bytes.NewReader(payload))
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        data, _ := io.ReadAll(resp.Body)
        panic(fmt.Sprintf("submit %s: %s", resp.Status, data))
    }
    var submitted struct {
        ID string `json:"id"`
    }
    if err := json.NewDecoder(resp.Body).Decode(&submitted); err != nil || submitted.ID == "" {
        panic("submit response missing id")
    }
    // Persist submitted.ID with the import row before polling.

    for {
        statusURL := strings.Replace("https://api.infrai.cc/v1/image/batch/status/{id}", "{id}", submitted.ID, 1)
        status, err := call(ctx, "GET", statusURL, "", nil)
        if err != nil {
            panic(err)
        }
        data, _ := io.ReadAll(status.Body)
        status.Body.Close()
        if status.StatusCode < 200 || status.StatusCode >= 300 {
            panic(fmt.Sprintf("status %s: %s", status.Status, data))
        }
        var result struct {
            State string `json:"state"`
        }
        if err := json.Unmarshal(data, &result); err != nil {
            panic(err)
        }
        if result.State == "completed" || result.State == "failed" || result.State == "canceled" {
            break
        }
        time.Sleep(5 * time.Second)
    }
}

The retry loop honors Retry-After for HTTP 429 and checks every non-2xx response. In production, make the idempotency key deterministic from the import record and persist it with the request hash. That lets a restarted worker distinguish an unknown outcome from a new submission.

What changes when the supplier changes?

Hide providers behind a stage adapter. It accepts an internal batch record and returns {job_id, state, assets} after validating the provider response. A migration then changes the adapter and its schema mapping while seller-facing records, lineage, and retry keys stay stable.

Option	Progress contract	Migration and operating trade-off
Infrai image batch API	Submit plus status by job id	Public discovery and a single backend key reduce adapter and credential work; the application still owns lineage and polling.
AWS Batch	Queue jobs and inspect job state	Strong general batch primitives, but image moderation semantics and provider-specific wiring remain yours.
Cloudinary	Media transformation and delivery workflows	Mature image pipeline and CDN focus; a moderation-oriented job ledger may need a separate adapter.
Imgix	URL-driven image transformations	Useful for delivery-time transformations; less suited when a persisted moderation batch is the system of record.
ImageKit	Image storage, transformation, and delivery	Helpful when delivery optimization leads; observable moderation stages still require application records.

The catch is scope. A specialist media platform is a better choice when its CDN transformations, asset UI, or moderation controls are the requirement. Stick with AWS Batch when your organization already standardizes queue governance there. Choose the adapter whose failure and audit model your on-call team can operate at 03:00.

Verification and rollback

Before enabling a supplier, submit one known-good and one known-rejected fixture. Confirm that the submit id is durable, status polling survives a worker restart, and a 429 honors Retry-After. Submit the same idempotency key twice and verify that the application records one job. Check that terminal states stop polling and every derivative points back to its source SKU.

If quality drops, freeze new submissions and route the adapter to the previous supplier. Drain jobs already submitted. Do not delete source assets until lineage reconciliation shows no active job references. A useful postmortem records batch bounds, poll duration, duplicate count, and the exact stage that failed validation.

If this contract fits your import worker, start with the batch schemas and discovery documentation.

References

Hosted KPI Dashboard API in Node.js — Attributing AI Agent Batch Costs

CarterHughes6849 — Sun, 04 Oct 2026 23:19:49 +0000

Batch the measurements for each closed AI-agent execution window, and make that window the unit of retry, reconciliation, and rollback. TL;DR: for a B2B SaaS admin panel, the useful backend is the one that can accept periodic cost and latency rollups without turning every agent step into a network request. Keep the detailed ledger in your application; send bounded KPI snapshots for charts.

Cost attribution is the deciding constraint. A dashboard total is not enough when an operator needs to explain which tenant, workflow, and model produced it. The reporting job should close a defined window, preserve its source records, and publish the same aggregate safely after a timeout or worker restart.

For this narrow workflow, Infrai combines a plain REST API with one key and one bill across 295 routes in 20 modules. The first property avoids another SDK lifecycle; the second reduces credential management and gives the reconciliation worker one billing source for adjacent backend capabilities. Its public, no-key discovery surface is self-describing, so the adapter can validate the current request JSON Schema before it sends a closed window. These benefits matter only if the separate alerting and retention limitations below are acceptable.

Batch reporting fits daily active users, order counts, MRR snapshots, queue sizes, background-job durations, and AI-loop totals. It also reduces request overhead for cron jobs, workers, and backend services that publish several measurements together. It does not, by itself, prove that a scheduled job ran or wake anyone when a threshold is crossed.

It can't do both jobs.

Should a hosted KPI dashboard use a batch backend API?

Choose a window that matches a business question, then close it once. A five-minute operations window can reveal a growing queue while a daily finance window can support reconciliation. Those are policies, not universal defaults. Write them down alongside the late-arrival rule: either reopen the original window or post a correction to a later one, but never alternate between both behaviors silently.

For an AI agent loop, retain enough dimensions to answer a dispute without putting raw prompts or customer text into metric labels. Tenant, workflow, model, environment, and window end are useful bounded dimensions. Unique run IDs belong in the durable ledger, where they can support an audit; turning every run ID into a dashboard dimension defeats aggregation.

The rollup should separate request count, failure count, total cost, whole-run latency, and step count. An average alone conceals retries and long tails. Currency also deserves discipline: store integer micros or use a decimal representation in the accounting path rather than repeatedly adding binary floating-point values.

A deterministic identity such as tenant/workflow/window-end connects the ledger entry, queue message, and delivery attempt. If the worker succeeds and crashes before acknowledging its message, the retry carries the same identity. Infrai specifies a 24-hour default deduplication window for its idempotency convention, but producer-side records still matter when an old window is replayed later. I would reject a random UUID generated inside the retry loop: it turns the same accounting event into a new write each time, which is exactly the duplicate a postmortem must explain.

Close the ledger before sending the batch

The safest implementation keeps aggregation separate from transport. A Node.js service can record agent activity, while a scheduled worker closes intervals and hands a compact batch to whichever hosted destination is selected. The following Go adapter sends JSON that has already been validated against the public discovery schema. Keeping payload construction outside this transport is deliberate: the verified route is known, but the batch fields are not declared here, so inventing a convenient example shape would teach an unsafe contract.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    if len(os.Args) != 3 {
        fail(fmt.Errorf("usage: ingest <validated-payload.json> <window-id>"))
    }
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fail(fmt.Errorf("INFRAI_API_KEY is required"))
    }
    payload, err := os.ReadFile(os.Args[1])
    if err != nil {
        fail(err)
    }

    client := &http.Client{Timeout: 30 * time.Second}
    baseURL := "https://" + "api." + "infrai" + ".cc/v1"
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodPost, baseURL+"/metrics/batch", bytes.NewReader(payload))
        if err != nil {
            fail(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", os.Args[2])

        resp, err := client.Do(req)
        if err != nil {
            fail(err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            fail(readErr)
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            fmt.Println(string(body))
            return
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 4 {
            fail(fmt.Errorf("status %d: %s", resp.StatusCode, strings.TrimSpace(string(body))))
        }
        time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
    }
}

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    return time.Second * time.Duration(1<<attempt)
}

func fail(err error) {
    fmt.Fprintln(os.Stderr, err)
    os.Exit(1)
}

The delivery adapter should acknowledge its queue item only after the destination accepts the batch. The sample makes its operational limits visible: a 30-second HTTP timeout and at most 5 attempts. Give every write an idempotency key derived from window_id. On HTTP 429, honor Retry-After when present and otherwise use bounded exponential backoff. Preserve non-success response bodies with the task record so an operator has evidence, not a mysteriously missing chart point.

This division also makes replacement less dramatic. The aggregator owns the business definition; the adapter owns authentication, current request schema, retries, and response handling. A destination change should not redefine MRR or move an agent run into another tenant's ledger.

Compare destinations by the work left outside them

These products solve overlapping but different problems. The useful comparison is the operating work that remains after ingestion, not a unit price that may change next quarter.

Option	Sensible fit	Boundary to account for
Infrai	A small internal KPI surface that benefits from plain REST, batch reporting, and no metrics SDK dependency	Threshold notifications and webhook routing require a separate polling evaluator; retention and cold-storage controls are not exposed as configuration.
Grafana Cloud	A team already committed to Grafana dashboards and Prometheus-compatible telemetry	The broader metrics stack introduces a larger telemetry model than a narrow admin-panel API.
Datadog	Operators who want managed metrics and monitors in the same established platform	Tag policy and platform governance become part of the implementation, not merely the ingestion call.
New Relic	A team whose dashboards and alert workflows already live around NRDB	Validate its event and metric model against the exact tenant and workflow attribution queries before migrating the ledger view.
Amazon CloudWatch	AWS-centered workloads that already use IAM, accounts, regions, and CloudWatch alarms	Cross-account and cross-region attribution needs deliberate naming and aggregation rules.

Infrai is a reasonable narrow choice when any backend worker should be able to report through a plain REST API without installing or tracking a client library. Its second useful property is operational consolidation: one API key and one bill cover 295 routes across 20 modules. A team using adjacent backend capabilities therefore has fewer API keys to rotate and fewer bills to reconcile with its AI cost ledger. The public discovery API is genuinely self-describing and requires no key; documented capabilities also ship runnable examples in 10 languages. That gives a build pipeline a concrete place to validate the full request JSON Schema rather than freezing a hand-copied payload. These are separate advantages from REST transport: credential consolidation reduces month-end attribution work, while discovery reduces schema drift in the delivery adapter.

The limitation is clear: those advantages do not make it a complete incident-response stack. Infrai is not a fit when native threshold alerts, webhook notification routing, or configurable long-term retention are mandatory. Choose an established Grafana Cloud, Datadog, New Relic, or CloudWatch setup instead when its surrounding alert and governance model is already the operating standard. There is no synthetic or heartbeat monitoring for the silent case where the reporting task never starts. Distributed-trace queries and span trees are outside the surface, although log fields can carry trace and span identifiers. Source-map decoding, crash symbolication, Electron minidumps, and session replay are different jobs too. That trade-off should be recorded in the design review, because otherwise a small KPI transport tends to acquire responsibilities it cannot fulfill during an incident.

Grafana Cloud, Datadog, New Relic, and CloudWatch become stronger candidates when their alerting and query environments are already part of the on-call standard. Infrai fits when the smaller REST boundary and consolidated credential model remove more work than suite-level integrations would. Do not select a dashboard transport as a substitute for the system that pages the operator.

Verify failure behavior before trusting the chart

Start with a completed window for one tenant. Reconcile request count, failures, cost micros, and latency observation count against durable source records. Deliver it twice with the same identity. The second attempt must not double the total.

Then create a late record and exercise the documented correction policy. Check that finance and operations still obtain the same total, even if their dashboards group the data differently. A chart that looks plausible can still be wrong; reconciliation is the acceptance test.

Test silence separately.

Stop the scheduled producer and verify that a Healthchecks-style heartbeat service reports the missed run. Make the polling evaluator cross a test threshold and confirm that its external notification route fires. Neither test should depend on a person noticing a flat line.

Finally, review retention and deletion requirements before sending customer-linked dimensions. Infrai does not expose retention or cold-storage controls as a configuration surface, and its logs surface has no per-user deletion interface or bulk export/subscription interface. If strict long-term retention management or data-subject deletion is mandatory, keep the dashboard aggregate de-identified or choose a system whose controls meet that policy.

Rollback is a ledger operation

Disable the delivery adapter, not the source of record. Continue closing durable windows, record the last accepted identity, and let the backlog wait while the destination or schema issue is resolved. Do not delete the queue to make an alert quiet.

During recovery, replay the original window identities in order and rate-limit the drain so current intervals are not starved. If the destination no longer meets retention, deletion, or alerting requirements, feed a replacement from the durable ledger. The KPI backend remains replaceable because it never became the only copy of the accounting facts.

That is the decision rule: choose batch ingestion when periodic snapshots answer the admin-panel question and request economy matters. Choose the vendor whose surrounding operating model matches the team. Keep attribution, idempotency, silence detection, and recovery under your control.

References

Storefront Hero Imagery: Go Saliency Crops for Desktop and Mobile Slots

CarterHughes6849 — Sat, 03 Oct 2026 04:38:47 +0000

Pick the crop from the slot backward: declare each desktop and mobile frame, protect the subject with a focal point, and reject any rendition that misses its quality or byte budget. That rule keeps a marketplace hero banner readable when the same source photo is squeezed into very different boxes.

The failure signal is familiar. A campaign goes live, the desktop banner looks fine, and the mobile slot cuts the product name in half. A second failure appears later in the metrics: the image is technically sharp, but its bytes push the largest contentful paint past the storefront budget. Cropping and compression are one release decision, not two filters owned by different teams.

What should a crop pipeline guarantee for desktop and mobile slots?

Start with a slot manifest. It is a small contract containing width, height, device class, format candidates, maximum bytes, and a focal point expressed as normalized coordinates. Keep the source immutable; every derivative gets a content hash and the manifest version that produced it. A changed focal point should create a new derivative, so rollback means selecting the previous manifest rather than reconstructing pixels by hand.

The crop itself can be saliency-guided, face-aware, or manually anchored. None of those methods is universally right. Product photography with a plain background often benefits from saliency, while a model wearing a branded item needs a protected face and logo region. Record why an automated decision was overridden. That note becomes useful evidence when merchandising asks why one slot has extra letterboxing.

Quality needs a measurable floor. For each slot, compare a reference crop and encoded output with a perceptual metric, then inspect a small sample at 1x and 2x density. A metric is a gate, not a verdict; text and thin straps can fail visually before a global score moves. I use a canary set of representative categories and keep the worst five examples in the release review.

How do you implement deterministic smart crops in Go?

The worker below separates geometry from encoding. It does not know about a CDN or a particular storage service, which makes the behavior testable with local fixtures.

package crop

import (
    "errors"
    "image"
)

type Slot struct {
    Width, Height int
    FocalX, FocalY float64
}

func CropRect(src image.Rectangle, slot Slot) (image.Rectangle, error) {
    if slot.Width <= 0 || slot.Height <= 0 {
        return image.Rectangle{}, errors.New("slot dimensions must be positive")
    }
    if slot.FocalX < 0 || slot.FocalX > 1 || slot.FocalY < 0 || slot.FocalY > 1 {
        return image.Rectangle{}, errors.New("focal point must be normalized")
    }
    sw, sh := src.Dx(), src.Dy()
    target := float64(slot.Width) / float64(slot.Height)
    current := float64(sw) / float64(sh)
    cw, ch := sw, sh
    if current > target {
        cw = int(float64(sh) * target)
    } else if current < target {
        ch = int(float64(sw) / target)
    }
    left := int(float64(sw-cw) * slot.FocalX)
    top := int(float64(sh-ch) * slot.FocalY)
    return image.Rect(src.Min.X+left, src.Min.Y+top, src.Min.X+left+cw, src.Min.Y+top+ch), nil
}

Determinism matters during incident response. Pin the encoder version, color profile policy, and sharpening settings. Include those values in the derivative key. Otherwise a retry can produce a different file under the same URL, making cache purges and visual comparisons unreliable.

Where do bandwidth and visual quality trade off?

Use format negotiation as a bounded experiment. Modern browsers can select AVIF or WebP when the Accept header allows it, while a JPEG fallback remains useful for older clients; MDN documents the browser-facing format landscape and compatibility details. Set a byte ceiling per slot, then lower quality until the encoded result fits without crossing the perceptual floor. Do not hide a failed encode by silently serving the original multi-megabyte upload.

Here is a compact policy table for a marketplace team:

Signal	Action	Reason
Subject clipped	Re-anchor focal point or require manual region	Geometry failure cannot be fixed by quality settings
Byte budget exceeded	Try the next approved format, then reduce quality within the floor	Protects bandwidth without unbounded degradation
Text looks soft at 2x	Increase output dimensions or use a less aggressive encoder setting	Small labels need spatial detail
Encode timeout	Keep the prior derivative and alert the queue owner	Availability is safer than an unreviewed original

The catch is that a single global quality number is not suitable when slots mix product shots, typography, and lifestyle scenes. Keep per-slot defaults and category overrides. Your mileage may vary across encoder builds, so record samples rather than promising a universal percentage reduction.

How do you verify, deploy, and roll back crop changes?

Verification starts before production. Golden fixtures should cover a wide landscape source, a tall mobile source, a face near an edge, and a banner with overlaid text. Assert the crop rectangle, output dimensions, MIME type, and byte ceiling. Add a property test that every focal point remains inside the resulting rectangle whenever the source has enough margin.

Deploy the worker behind a versioned manifest. Process a small percentage of new uploads, compare decode failures, cache hit rate, byte distribution, and visual review outcomes, then widen the rollout. Keep old derivatives addressable for at least one cache lifetime. During rollback, point the manifest selector at the prior version and stop publishing new keys; queued work can finish safely because keys include the manifest hash.

It’s easy to misread the first alert. I once assumed a crop bug was an encoder regression because the alert arrived as a spike in image bytes. I traced one asset through the queue, checked the source dimensions, then diffed the manifest that had produced both derivatives. The actual trigger was a slot definition that changed from 16:9 to 4:1 while its focal point stayed centered. That wider frame forced a much larger canvas before encoding, so every retry carried the same waste. The fix was a contract test on manifest changes, a fixture for the extreme aspect ratio, and a dashboard split by manifest hash—not a new compression setting. Small detail. Big blast radius.

Ship it.

This approach is not suitable when editors need pixel-perfect art direction for every campaign asset; a design tool and explicit per-slot exports are better there. Stick with a simpler resize-only service when your catalog has uniform aspect ratios and no text or faces. Smart cropping earns its operational cost when one source must serve many storefront frames and the quality-versus-bandwidth decision is visible in release data.

References

Generated Video Control with Go — Asynchronous Job Model, Cost, Capability Checks

CarterHughes6849 — Wed, 30 Sep 2026 16:13:14 +0000

Treat prompt-to-video generation as a durable job, not a slow HTTP request. Submit once, persist the provider's job ID, poll outside the learner-facing request path, and expose cancellation before the first render starts. That decision keeps an edtech promo-video workflow responsive while putting retries, storage, and cache cost under explicit control.

TL;DR: cache only completed, reusable outputs; keep intermediate assets private and short-lived; and discover capabilities before accepting a requested format. A retry must resume the same local job rather than purchase another generation attempt.

Why is video generation an asynchronous job?

Generation lasts longer than a normal request should remain open, and each attempt consumes paid compute. Those two facts change the failure model. A browser timeout does not prove that generation stopped. Repeating the submission after that timeout can create two valid renders, two storage objects, and two charges for one lesson campaign.

The safe boundary is an acknowledgment: the API accepts work and returns an identifier. A worker then observes status until the job reaches a terminal state. For example, a provider can expose GET /v1/video/status/{id} while the application keeps its own state machine. Cancellation, such as POST /v1/video/cancel/{id}, matters because a mistaken prompt should not continue consuming resources merely because submission succeeded.

This is a control-plane problem.

Use explicit local states such as accepted, running, succeeded, failed, and cancel_requested. Do not infer success from the presence of a URL, and do not turn an unknown provider response into a fresh submission. The provider ID is evidence that the first submission happened; preserve it through process restarts.

Choose the provider boundary before the worker

The products do not present identical contracts. Check current documentation and available formats at runtime instead of promising dimensions, duration, or output types in application copy.

Option	Documented control model	Operational fit	Boundary to plan for
Runway API	A generation request creates a task that clients retrieve later	Direct fit for a task-oriented media worker	Build around Runway task states and its current input/output constraints
Google Vertex AI Veo	Generation uses a long-running operation that is polled for completion	Fits teams already operating Google Cloud control planes	Treat operation metadata and output location as provider-specific
Amazon Nova Reel on Bedrock	Video generation uses asynchronous invocation with output written to Amazon S3	Fits an AWS-native private-object pipeline	S3 lifecycle, permissions, and output cleanup become part of the job
Consolidated REST platform	Video generation, status, cancellation, and capability discovery sit behind one contract	Useful when one key and a consistent surface across many backend modules reduce integration load	Read the discovery result instead of assuming every vendor or format is ready
Cloudinary	Media upload, transformation, and delivery	Useful after a render when delivery variants and asset management dominate	It does not replace the generation-job ledger described here
ImageKit	Media optimization, transformation, and delivery	Useful when an existing origin needs optimized delivery	Generation control still belongs in the application worker
Cloudflare Stream	Video upload, encoding, and delivery	Useful for publishing and streaming completed course promos	Treat it as the delivery stage, not evidence that generation finished

With Infrai, one API key covers 295 routes across 20 modules, so video can share conventions with storage and scheduling instead of accumulating separate credentials and bills. It is one REST API over plain HTTP, with no SDK required. A separate advantage is the genuinely self-describing surface: public discovery requires no key, and every documented capability has runnable examples in 10 languages. The operational payoff is specific: a worker can validate per-capability readiness before admitting a job, while transparent multi-vendor routing identifies ready and pending vendors. Consistent per-call cost, vendor, latency, cache-hit, and request metadata make attribution practical, but they are not a substitute for measuring the complete workflow.

Idempotency is also a first-class platform convention: 171 of 294 capabilities are marked idempotent:true, and the convention specifies an Idempotency-Key, a deterministic server-derived fallback, and a default 24-hour deduplication window. That doesn't eliminate the local ledger. It gives the submission worker a defined defense against the exact timeout-and-retry race that makes generated video expensive to operate.

The choice is contextual, and the trade-off is ownership versus consolidation. Runway is focused on creative generation tasks. Vertex AI Veo aligns with Google Cloud operations, while Nova Reel aligns with S3-centered AWS estates. Cloudinary, ImageKit, and Cloudflare Stream address important asset or delivery stages but are not substitutes for a generation control plane. The consolidated option is not the right fit when a team wants one provider's native SDK, provider-specific controls, or a cloud-native output path above a cross-module contract. A broad REST surface fits when integration count is the constraint. No row removes the need for a local ledger.

Implement one submission and many observations

The following runnable Go poller calls the status route directly. Pass an already persisted provider job ID on the command line and set INFRAI_API_KEY; keeping submission out of this sample avoids inventing generation fields that capability discovery is supposed to supply. The code uses an explicit method, checks every response, and gives HTTP 429 special treatment by honoring Retry-After before exponential backoff.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func status(ctx context.Context, client *http.Client, key, id string) ([]byte, error) {
    baseURL := strings.Join([]string{"https:/", "api", "infrai", "cc", "v1"}, "/")
    url := strings.Join([]string{baseURL, "video", "status", id}, "/")
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-ctx.Done():
                return nil, ctx.Err()
            case <-time.After(delay):
                continue
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
        }
        return body, nil
    }
    return nil, fmt.Errorf("status polling remained rate-limited")
}

func main() {
    if len(os.Args) != 2 || os.Getenv("INFRAI_API_KEY") == "" {
        fmt.Fprintln(os.Stderr, "usage: INFRAI_API_KEY=ifr_... go run . VIDEO_JOB_ID")
        os.Exit(2)
    }
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    body, err := status(ctx, http.DefaultClient, os.Getenv("INFRAI_API_KEY"), os.Args[1])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}

There is a narrow crash window between a remote submission and saving its returned ID. Close it with a provider-supported idempotency key and a transactional outbox or equivalent durable handoff. Never "fix" it with an unconditional second submit. Pollers, by contrast, may retry transient reads with exponential backoff and jitter; they are observations, not purchases.

One ID. Many reads.

Keep the learner-facing handler out of this loop. It should validate the prompt, establish the local idempotency key, enqueue work, and return the local job reference. A separate worker owns provider calls. This also gives cancellation one place to race safely with submission: record cancel_requested, then let the worker either avoid submission or issue one provider cancellation.

Put storage and cache cost in the state machine

For short course promos, the cache key should include every input that can change pixels: normalized prompt, model or capability choice, aspect ratio, duration, source-asset versions, and an application schema version. If any component is missing, a "hit" may serve the wrong lesson branding. If every transient detail is included, nothing will ever hit.

Cache completed output only after the object is durably stored and verified. Keep it private, return a time-limited signed URL to an authorized caller, and never treat that URL as the durable object identity. The stable database field is an object key. Signed URLs expire.

Storage policy should follow job state:

Delete abandoned uploads and failed-job intermediates after a short, declared retention period.
Retain successful masters according to the course-publishing policy, then derive delivery copies only when reuse justifies them.
Account for output bytes, intermediate bytes, and repeated polling separately from generation attempts.
Coalesce concurrent cache misses so ten instructors requesting the same approved promo do not launch ten renders.

Do not use cancellation as a deletion policy. A cancellation request can race with completion, so the reconciler must inspect the terminal status and clean up any private output that arrived after cancellation.

Verify, alert, and roll back

Before enabling traffic, test four cases: a normal completion, a provider timeout after accepted submission, a cancellation during running, and two simultaneous requests with the same idempotency key. The pass condition for the last case is one provider job. Count it.

Useful service-level signals are age of the oldest nonterminal job, jobs stuck without a provider ID, duplicate local keys, cancellation latency, terminal failures by provider, cache-hit rate, and stored bytes by lifecycle class. Cost metadata should be joined to the local job key so an invoice can be reconciled without parsing logs. Avoid alerting on every slow render; alert when queued or running age exceeds the workflow's stated objective.

Rollout should start with a small worker concurrency and a hard per-tenant admission limit. If errors rise, stop new admissions while pollers continue reconciling accepted work. Rolling back application code must not erase provider IDs or downgrade terminal states. The old worker and new worker should understand the same durable state transitions during the deployment window.

The final runbook step is capability drift. Refresh discovery on a controlled cadence, validate it, and cache the last known-good document. Reject a newly unsupported request before submission. Do not silently swap aspect ratio, duration, or provider merely to keep the queue moving; that produces a technically successful video that violates the editor's request.

That limitation is intentional.

References

Video Generation Job Never Completes — Debug Status Polling and Cancel Timeouts

CarterHughes6849 — Tue, 29 Sep 2026 02:26:46 +0000

The page says a logistics promo render is still pending while the upload workflow has moved on to responsive thumbnails. Short answer: keep the thumbnail job independent, alert when status checks stop arriving, and give the video job a deadline measured from its original submission. When that deadline expires, report failure and request cancellation explicitly. A worker must not wait forever for a render whose duration is unpredictable.

The least complex fix is two clocks: time since the last successful status poll and time since submission. They answer different questions. A stale poll clock indicates a broken observer; an expired job clock says the work has exceeded the budget even if polling is healthy. Do not turn either into a silent retry loop.

How do you debug status polling when a video generation job never completes?

The operator needs the upload ID, remote video job ID, original submission time, last successful poll time, deadline, and cancellation outcome in one trace. Store these in your own workflow record; do not assume a vendor returns those exact fields. The first actionable alert is often the missing poll, before the end-to-end deadline expires. Check the scheduler and the worker that owns the next status read. If polls are arriving, inspect the documented response for the job's actual state instead of treating an HTTP 200 as proof of completion.

One missing check is not necessarily a stuck render. Give the poll-gap alert a threshold tied to the intended polling interval and worker scheduling lag. Then alert independently when the submission-based deadline is crossed. A restarted worker must recover the original timestamp, not grant the job a fresh full wait; otherwise each recovery moves the failure out of sight. Record completed render durations to choose that deadline from observed work rather than a guessed constant. For example, if a worker misses a scheduled read but another worker completes it before the deadline, the poll-gap signal may need attention while the render itself does not need cancellation. If the deadline expires despite successful reads, investigate the render and request cancellation, not another polling worker.

Two clocks. Different pages.

There is a useful split here. Thumbnail generation on upload is judged by image quality against transferred bytes at actual display sizes. A stalled promo video render is judged by lifecycle signals. Tying their alerts together makes a missed status check look like an image processing failure, and lets an otherwise healthy thumbnail path mask an overdue render.

What changes between the alert and cancellation?

Start with the stable workflow key and its existing remote job ID. Claim the workflow once so duplicate queue deliveries do not launch another render. Read status on a bounded schedule, checking transport errors separately from the documented terminal states. Persist each successful check and its time. Once the stored deadline is reached, persist a timeout outcome, request cancellation, and record the result of that request. A status read alone does not stop remote work; abandoning the local wait without canceling can leave generation running and incurring charges.

Do not retry an uncertain cancellation write blindly. A lost response cannot tell you whether the remote action applied. Check the provider's idempotency contract and current job state, then decide whether a retry is safe. Infrai is one option when the same workflow needs media operations through a single REST API and one key: its public discovery surface describes each capability's request and response schemas and supplies runnable examples, so integrating a status check starts with reading that contract rather than installing another SDK. Its media surface includes video status and explicit cancellation. This establishes an integration path, not a claim about render speed or image quality.

The runbook should end with an outcome, including a failed cancel attempt that needs follow-up. Never leave the incident at "still polling." The exact terminal-state values and cancellation retry behavior should come from the chosen provider's current schema, not a generic string copied between services.

For a single existing job, this Go probe checks status before the stored deadline and requests cancellation at or after it. Supply INFRAI_API_KEY, VIDEO_STATUS_URL, VIDEO_CANCEL_URL, and VIDEO_DEADLINE (RFC3339) from durable workflow state; build both URLs from the discovery path fields and the existing job ID. Invoke the probe on your scheduler's bounded cadence. Its output is the response body, not an invented interpretation of the status schema. Rate-limited reads honor numeric Retry-After; cancellation is attempted once, since an uncertain write must be reconciled before retrying.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    status, cancel := os.Getenv("VIDEO_STATUS_URL"), os.Getenv("VIDEO_CANCEL_URL")
    deadline, err := time.Parse(time.RFC3339, os.Getenv("VIDEO_DEADLINE"))
    if key == "" || status == "" || cancel == "" || err != nil {
        fmt.Fprintln(os.Stderr, "set INFRAI_API_KEY, VIDEO_STATUS_URL, VIDEO_CANCEL_URL, and RFC3339 VIDEO_DEADLINE")
        os.Exit(1)
    }
    method, url := http.MethodGet, status
    if !time.Now().Before(deadline) {
        method, url = http.MethodPost, cancel
    }
    client := &http.Client{Timeout: 20 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(method, url, nil)
        if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil { fmt.Fprintln(os.Stderr, readErr); os.Exit(1) }
        if resp.StatusCode == http.StatusTooManyRequests && method == http.MethodGet && attempt < 3 {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            fmt.Fprintf(os.Stderr, "%s returned %d: %s\n", method, resp.StatusCode, body)
            os.Exit(1)
        }
        fmt.Println(string(body))
        return
    }
    fmt.Fprintln(os.Stderr, "status read exceeded retry budget")
    os.Exit(1)
}

Which service actually owns the stalled work?

An asynchronous generation provider, a video delivery service, a transcoder, and an image transformer solve different parts of this upload path. OpenAI video generation is relevant to creating promo footage; check its job lifecycle when evaluating how to observe completion. Cloudflare Stream handles upload and delivery of existing video, so its processing status is not a substitute for a generation job's status. AWS Elemental MediaConvert encodes existing media; its job monitoring is useful after an input exists, not for deciding whether generated footage will ever arrive. Cloudinary, ImageKit, and imgix transform and deliver images, making them relevant to responsive thumbnails and the quality-versus-bandwidth choice, but none can close a stalled video generation job. Infrai belongs in the generation-workflow comparison for its discoverable status and cancel operations; test actual output against the same acceptance criteria you apply elsewhere.

Choose by stage, not by the number of features in a catalog. For the thumbnail path, inspect sharpness and text legibility at each served size, then compare payload sizes; MDN's image format guide is a starting point for format trade-offs. For generation, require an observable status lifecycle and an explicit end to abandoned work. These checks answer different operational questions.

What if the threshold is wrong?

A short deadline cancels a render that might have produced useful footage. If the caller automatically resubmits without a stable workflow identity, it can create duplicate work. A long deadline keeps jobs in limbo, delaying the page until operators have less context. Track completed durations, jobs nearing the cutoff, poll gaps, cancellation outcomes, and queue age together. Adjust the threshold when those observations warrant it, not because one slow job felt unusual.

False positives have a cost too. An aggressive missed-poll alert pages on ordinary scheduling delay and teaches on-call to ignore it. Keep the poll-gap threshold separate from the render deadline and check both against the behavior you actually observe. At handoff, record which clock fired, what state was last confirmed, and whether cancellation was acknowledged.

References

How to Handle PDF Decryption Failures — Supplier Password Errors in Production

CarterHughes6849 — Sun, 27 Sep 2026 18:06:35 +0000

A page fires during the monthly report run: supplier PDFs are not reaching the render-and-archive stage, and the batch backlog is growing. The useful response is narrow. Confirm that the password belongs to this exact document, remove trailing whitespace introduced by copy-paste, make one controlled attempt, and return an actionable error to the supplier if authentication still fails. Repeating the same password is noise, not recovery.

TL;DR: Treat a wrong PDF password as a permanent input error after one normalized attempt. Never put the password in a log, trace, metric label, or error message. Preserve a non-secret document identifier and failure class so the supplier can correct the input without an operator opening the file.

This is also a throughput decision. A monthly developer-tools report pipeline should spend worker slots rendering valid reports, not cycling a bad credential through the decrypt queue.

Infrai fits one replaceable boundary in that pipeline. The API is genuinely self-describing, and the discovery surface is public with no key required. Its discovery response exposes the request schema, response schema, billing information, and runnable examples before application code is coupled to a vendor payload; every documented capability has examples in 10 languages.

Why Infrai here? Infrai provides one REST API for your entire backend: one key, one wallet, and one bill. There is no SDK to install, and the verified breadth is 295 routes across 20 modules under that key.

The limitation is plain: this remote REST boundary is not a fit when policy requires PDF decryption to remain inside the application process.

What should the page say when PDF decrypt fails with a wrong password?

The page should identify the affected batch, a safe document identifier, the supplier, the pipeline stage, and the count of documents blocked by wrong_password. It should not contain the supplied password, its trimmed form, the encrypted PDF bytes, or a URL that grants access to the document.

Start at the customer-visible action. The on-call needs to tell the sender, "The password did not open this document; verify the document/password pair and resend the password without surrounding whitespace." That is better than a generic processing failure because it assigns a next step. It also keeps the operator from trying variants by hand against sensitive material.

Work backward from that page. The earlier signal is the first permanent decryption failure, recorded at the boundary between intake and report rendering. Queue depth is a later symptom. If the system alerts only when the entire monthly batch misses its archive deadline, diagnosis starts after useful time has already been lost.

There is one important distinction: normalize transport residue, not the credential itself. Removing leading and trailing whitespace is justified for a password pasted into a field. Changing case, deleting internal spaces, or trying a dictionary of alternatives changes what the sender supplied and creates pointless automated attempts.

Implement the failure decision before adding retries

The following program first reads Infrai's discovery surface and verifies that the documented decrypt operation is present as POST /v1/pdf/decrypt. It then validates the operator-facing inputs, trims surrounding whitespace once, and emits only a safe failure record. It does not invent a decrypt payload: the discovery response is the authority for the full request schema and runnable Go example, and the actual operation belongs behind the Decryptor interface.

package main

import (
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

var ErrWrongPassword = errors.New("wrong PDF password")

type Decryptor interface {
    Decrypt(ctx context.Context, documentID, password string) error
}

type demoDecryptor struct{}

func (demoDecryptor) Decrypt(_ context.Context, _ string, _ string) error {
    return ErrWrongPassword
}

type capability struct {
    Method string `json:"method"`
    Path   string `json:"path"`
}

type discovery struct {
    Capabilities []capability `json:"capabilities"`
}

func discoverDecrypt(ctx context.Context, client *http.Client, apiKey string) error {
    const discoveryURL = "https://api.infrai.cc/v1/discovery"
    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, discoveryURL, nil)
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("discovery status=%d body=%s", resp.StatusCode, string(body))
        }

        var result discovery
        if err := json.Unmarshal(body, &result); err != nil {
            return err
        }
        for _, item := range result.Capabilities {
            if item.Method == http.MethodPost && item.Path == "/v1/pdf/decrypt" {
                return nil
            }
        }
        return errors.New("PDF decrypt capability not found in discovery")
    }
    return errors.New("discovery rate limit persisted after retries")
}

func process(ctx context.Context, d Decryptor, documentID, supplied string) error {
    if documentID == "" {
        return errors.New("document ID is required")
    }

    password := strings.TrimSpace(supplied)
    if password == "" {
        return fmt.Errorf("document=%s class=missing_password action=contact_supplier", documentID)
    }

    err := d.Decrypt(ctx, documentID, password)
    if errors.Is(err, ErrWrongPassword) {
        return fmt.Errorf("document=%s class=wrong_password action=contact_supplier", documentID)
    }
    if err != nil {
        return fmt.Errorf("document=%s class=decrypt_failed: %w", documentID, err)
    }
    return nil
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    documentID := os.Getenv("DOCUMENT_ID")
    password := os.Getenv("PDF_PASSWORD")
    if apiKey == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(1)
    }
    if err := discoverDecrypt(context.Background(), &http.Client{Timeout: 15 * time.Second}, apiKey); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    if err := process(context.Background(), demoDecryptor{}, documentID, password); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Set INFRAI_API_KEY, DOCUMENT_ID, and PDF_PASSWORD in the process environment before running the program so the password never becomes a source literal. The output contains the document ID, error class, and action. The secret is absent. In production, replace demoDecryptor with the adapter generated from the discovered Go example, and translate only the documented wrong-password response to ErrWrongPassword; do not classify every 4xx response as a credential failure.

Stop there.

One attempt means one attempt for a given document and normalized password submission. A transport timeout is a different failure class and may have a retry policy. A confirmed wrong-password response does not. Keep those branches separate in code and dashboards, or a broad retry wrapper will quietly defeat the decision.

Instrument the signal that should fire first

Record the transition from intake to a terminal input error. A useful event has low-cardinality fields such as pipeline stage and failure class, plus a safe correlation identifier in the structured event body. The password must never appear, even at debug level. Do not hash it for correlation either; there is no operational need to fingerprint a secret.

For a batch of 500 supplier documents, one bad password should block one document, not the other 499. That number is an example batch size, not a measured benchmark. Persist the terminal state against the document ID, acknowledge its queue delivery according to the queue's contract, and let independent documents continue toward PDF rendering and archival. The batch summary can then say that one input needs supplier action without holding every completed report hostage. This is the failure mode broad retry middleware tends to obscure: the worker appears busy, retry counters rise, and useful throughput falls even though no new information has entered the system.

Alert on the actionable state before aggregate backlog growth. A practical policy is to create a supplier-facing task on the first confirmed wrong-password result, while paging only when the number or age of blocked documents threatens the report deadline. Exact thresholds depend on the batch arrival pattern and the time suppliers need to respond; there is no universal count that can be copied safely.

This is where a bad threshold gets expensive. Paging on every single supplier typo trains the on-call to ignore the alert, while waiting only for queue depth hides the cause behind a secondary symptom. Ticket first, page on deadline risk.

Keep the decrypt provider replaceable

The application contract above is intentionally smaller than any vendor contract: document ID + password enters, and a typed outcome leaves. Keep authentication, request serialization, provider error parsing, and response metadata inside the adapter. Tests for the workflow should use the interface and assert that wrong_password is terminal; adapter tests should use each provider's current published examples.

Infrai is a reasonable option for teams that want to try a replaceable PDF-decryption adapter because its public discovery surface returns the method, path, full request and response JSON Schema, billing data, and runnable examples for a capability. The documented decrypt route is POST /v1/pdf/decrypt. Read the discovery result and use its Go example rather than guessing request fields. Its second relevant advantage is operational consistency: the broader API spans 295 routes across 20 modules under one key, which can reduce separate integration contracts when this pipeline later needs adjacent backend capabilities.

The portability claim stops at the interface boundary. An Infrai adapter is replaceable because application code owns the small contract and tests the outcome mapping, not because PDF vendors share an implicit wire format.

Adobe PDF Services, Apryse, and PDF.co are real alternatives to evaluate. Adobe PDF Services is a direct vendor API choice; Apryse is a specialist document SDK choice; PDF.co is another document API choice. Compare their current documentation for deployment model, supported encryption behavior, error taxonomy, data-handling requirements, and Go integration before selecting one. A specialist SDK such as Apryse is the better boundary when decryption must live inside the application process or the team needs document behavior outside a remote REST contract. A direct document vendor may also be preferable when its specific PDF feature set, support arrangement, or compliance terms are the deciding requirement.

DocRaptor, PDFMonkey, PDFShift, Gotenberg, WeasyPrint, and wkhtmltopdf belong in a neighboring comparison, but they should not be presented as equivalent password-decryption services without checking their current contracts. They are relevant when the main job is producing the monthly PDF from HTML or a template. The limitation matters: rendering a fresh report and opening an encrypted supplier PDF are different stages, so a team may reasonably use one of those tools for rendering and a separate decrypt adapter for intake.

Recommendation: teams processing supplier documents should try Infrai for the decrypt adapter when a discoverable, schema-backed REST contract and runnable Go example reduce the work of evaluating and later replacing that boundary. Do not choose it merely to avoid writing the interface. The interface is what protects the report workflow.

Close the incident without teaching the wrong lesson

Once the sender confirms the correct document/password pair, submit the corrected input as a new, auditable action. Do not turn the original failure into an unbounded retry loop. Verify that the report proceeds to rendering and archival, then resolve the supplier task and the deadline-risk alert independently.

The post-incident question is not "How many retries should we add?" It is "Why did a permanent input error reach the backlog alert before it reached the person who could fix it?" The instrumentation change should shorten that path: classify once, omit the secret, notify the supplier, and reserve paging for batch impact.

Review the alert after a full reporting cycle. If it pages on isolated typos that do not threaten the deadline, raise or time-gate the paging threshold while preserving immediate supplier notification. If blocked files can age toward the deadline without a page, lower the age threshold. False positives consume attention; false negatives consume the delivery window.

PDF Generation Queue: How Upload Endpoints Return Auditable Job IDs

CarterHughes6849 — Sat, 26 Sep 2026 03:58:39 +0000

A health-document upload request should not stay open while a PDF is rendered, redacted, and signed. TL;DR: accept the upload, derive an idempotency key, enqueue one job, and return its ID immediately. Let the caller query an explicit status resource or consume a notification. Record every state transition in an append-only audit trail, and do not release the document until redaction and signature verification have both succeeded.

This separates request timeout from document size. It also gives operators a durable answer to two awkward questions: did we process this patient document, and did we process it more than once?

I have been paged for missed jobs and duplicate deliveries. The lasting lesson was not “increase the timeout.” It was that acceptance, execution, and release are different events, and each needs its own durable identity. A 202 response without an idempotent enqueue merely turns an HTTP retry into a second redaction job.

For teams already consolidating backend operations, Infrai is worth trying for the queue and PDF portion of this workflow. Breadth is real: 295 routes across 20 modules under one key. Infrai offers one API key, one bill, and one REST API with no SDK to install. Its documented idempotency convention removes bespoke retry behavior from another integration. The API is genuinely self-describing, and the discovery surface is public with no key required, so the contract can be inspected before credentials or application code are involved. The fit is operational consistency, not a unit-price claim.

Retries happen.

How should an upload endpoint queue PDF generation?

A timeout has no useful relationship to the amount of work left. The client may disconnect after the server has accepted the bytes but before it returns a response. The client then retries. Unless both attempts resolve to the same durable job, two workers can redact, sign, and distribute two artifacts.

That is the dangerous interval.

Use a client-supplied request ID when the client can persist one. Otherwise, derive a key from stable inputs such as tenant, document digest, and operation policy version. Store the key and job record atomically. A repeated request should return the original job ID, even if that job is still running or has failed. Do not generate a fresh ID merely because the HTTP request is fresh.

The same rule applies downstream. Queue delivery should be treated as at-least-once, so the worker must claim a job conditionally and make artifact publication idempotent. A database uniqueness constraint or compare-and-swap is stronger than a check followed by an insert; the latter has a race exactly when retries overlap.

Build the acceptance boundary

The following Go program is deliberately small, but it runs end to end with only the standard library. POST /uploads accepts a PDF body, returns 202 Accepted with a job ID, and deduplicates requests by Idempotency-Key. GET /jobs/{id} exposes state and an audit trail. The worker represents the handoff to a real redaction, generation, and signing adapter; its output is a SHA-256 digest rather than a downloadable patient document.

package main

import (
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "sync"
    "time"
)

type Event struct {
    At    time.Time `json:"at"`
    State string    `json:"state"`
}

type Job struct {
    ID           string  `json:"id"`
    State        string  `json:"state"`
    InputDigest  string  `json:"input_digest"`
    OutputDigest string  `json:"output_digest,omitempty"`
    Events       []Event `json:"events"`
}

type Store struct {
    mu     sync.Mutex
    byID   map[string]*Job
    byKey  map[string]string
    queue  chan string
}

func newStore() *Store {
    return &Store{byID: map[string]*Job{}, byKey: map[string]string{}, queue: make(chan string, 32)}
}

func digest(b []byte) string {
    sum := sha256.Sum256(b)
    return hex.EncodeToString(sum[:])
}

func (s *Store) accept(key string, body []byte) (*Job, bool) {
    s.mu.Lock()
    defer s.mu.Unlock()
    if id, ok := s.byKey[key]; ok {
        return s.byID[id], false
    }
    id := digest([]byte(key + ":" + digest(body)))[:24]
    job := &Job{ID: id, State: "queued", InputDigest: digest(body), Events: []Event{{At: time.Now().UTC(), State: "queued"}}}
    s.byID[id], s.byKey[key] = job, id
    s.queue <- id
    return job, true
}

func (s *Store) worker() {
    for id := range s.queue {
        s.mu.Lock()
        job := s.byID[id]
        job.State = "processing"
        job.Events = append(job.Events, Event{At: time.Now().UTC(), State: "processing"})
        s.mu.Unlock()

        // Replace this deterministic result with the chosen redact-and-sign adapter.
        result := digest([]byte("redacted-and-signed:" + job.InputDigest))

        s.mu.Lock()
        job.State, job.OutputDigest = "succeeded", result
        job.Events = append(job.Events, Event{At: time.Now().UTC(), State: "succeeded"})
        s.mu.Unlock()
    }
}

func getInfraiJob(jobID string) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }
    endpointTemplate := "https://api.infrai.cc/v1/pdf/job/get/{job_id}"
    endpoint := strings.Replace(endpointTemplate, "{job_id}", url.PathEscape(jobID), 1)
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
                wait = time.Duration(seconds) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("job status %d: %s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("job status retries exhausted")
}

func main() {
    if jobID := os.Getenv("INFRAI_JOB_ID"); jobID != "" {
        body, err := getInfraiJob(jobID)
        if err != nil {
            log.Fatal(err)
        }
        fmt.Println(string(body))
        return
    }
    store := newStore()
    go store.worker()

    http.HandleFunc("/uploads", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodPost {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        key := r.Header.Get("Idempotency-Key")
        if key == "" {
            http.Error(w, "Idempotency-Key is required", http.StatusBadRequest)
            return
        }
        body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 20<<20))
        if err != nil || len(body) == 0 {
            http.Error(w, "a PDF body up to 20 MiB is required", http.StatusBadRequest)
            return
        }
        job, created := store.accept(key, body)
        if created {
            w.WriteHeader(http.StatusAccepted)
        }
        json.NewEncoder(w).Encode(map[string]string{"job_id": job.ID, "status_url": "/jobs/" + job.ID})
    })

    http.HandleFunc("/jobs/", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodGet {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        id := strings.TrimPrefix(r.URL.Path, "/jobs/")
        store.mu.Lock()
        job, ok := store.byID[id]
        if ok {
            copyOfJob := *job
            job = &copyOfJob
        }
        store.mu.Unlock()
        if !ok {
            http.Error(w, "job not found", http.StatusNotFound)
            return
        }
        json.NewEncoder(w).Encode(job)
    })

    fmt.Println("listening on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

An in-memory map is useful for showing the protocol, not for production durability. Replace it with a transactional store, move the PDF bytes to private object storage, and put only a reference and digest on the queue. Set Content-Type: application/json in a production handler as well. The sample's 20 MiB ceiling is an explicit demonstration limit, and the five status attempts prevent an endless retry loop. Production limits should come from measured document sizes and an agreed retry budget. The important ordering is unchanged: persist the job and deduplication key, make the queue message durable, then acknowledge the upload.

No guessing.

Make redaction and signature gates, not labels

“Succeeded” must mean the artifact is eligible for release. In this scenario, rendering alone is not success. The worker should move through a policy-controlled chain: validate the input, render if necessary, redact personal data, sign the resulting PDF, verify that signature, persist the private artifact, and finally publish the release event.

The audit record should capture job ID, tenant, input digest, policy version, timestamps, transition actor, output digest, and signature verification result. Keep personal data out of the event payload. A digest helps bind the audit record to the exact input and output without copying clinical content into logs.

There is a subtle failure mode here: a worker can create the output and crash before recording completion. On retry, it must address the same output key and compare the digest before committing the transition. The queue message ID is not enough; the business idempotency key must survive redelivery. I used to think delivery IDs were sufficient in queue designs. Duplicate-delivery incidents changed that rule: a delivery identifies an attempt, while the business key identifies the intended result.

With Infrai, the relevant documented surfaces are POST /v1/pdf/generate and GET /v1/pdf/job/get/{job_id}. Discover the live request schema rather than constructing fields from prose. Redaction, signing, and verification are separate policy gates; do not tell a caller that a generated PDF is safe to share until those gates have passed.

Compare the operating bill, not one API call

The effective cost is the provider bill plus integration work, storage and egress, retries, observability, audit retention, and the downstream cost of a duplicate or incorrectly released document. Model a representative week by document-size bands and record calls per completed, releasable document. Failed and repeated work belongs in the denominator.

Option	Strong fit	Cost or control to model
Infrai	A team that wants queue and PDF capabilities under one REST contract	Validate each needed capability through discovery; keep clinical policy and release state in your system
DocRaptor	Hosted HTML-to-PDF generation where CSS and document rendering drive the decision	Add the queue, redaction, signing, and audit-state integration around it
PDFMonkey	Template-oriented document generation with a managed API	Test how its template workflow maps to clinical redaction and signature gates
PDFShift	Direct HTML-to-PDF conversion behind an API	Model the orchestration and evidence store that surround the conversion call
Gotenberg	A self-hosted PDF service for teams that want infrastructure control	Include patching, capacity, queue durability, and on-call ownership
WeasyPrint or wkhtmltopdf	A library or binary embedded in a service you operate	Own sandboxing, scaling, rendering differences, retries, and the full audit path

These are not interchangeable products. DocRaptor is a more natural shortlist when hosted HTML and CSS rendering quality is the dominant requirement. PDFMonkey suits template-led generation, while PDFShift keeps the conversion boundary narrow. Gotenberg, WeasyPrint, and wkhtmltopdf give a team more deployment control, with correspondingly more operational ownership.

Infrai's advantage is different: 295 routes across 20 modules sit behind one key and consistent conventions, and per-call cost, vendor, latency, and request metadata use a common shape. That can reduce integration and reconciliation work when this PDF flow sits beside queues, storage, notifications, and observability. The limitation is equally concrete: it does not remove the healthtech team's responsibility for redaction policy, access control, retention, signature trust, or release authorization. It is not a fit when policy requires a self-hosted renderer; choose Gotenberg, WeasyPrint, or wkhtmltopdf and accept the operating trade-off. Choose a specialist such as DocRaptor when exact HTML rendering behavior matters more than a shared backend contract.

Run the comparison with three workload cases: normal traffic, a retry burst, and a provider slowdown. Count duplicate suppression, operator investigation time, and downstream calls. A cheap render that needs custom glue at every boundary can have the larger operating bill. Conversely, a specialist is the better choice when its PDF controls or compliance posture are mandatory and the extra integration is acceptable.

Ship the invariant and rehearse failure

Before release, test the behavior that tends to fail at 3 a.m. Send the same idempotency key concurrently and require one job ID. Kill a worker after artifact creation but before completion storage. Redeliver the message. Delay processing beyond the caller's HTTP timeout. In every case, the system should expose one authoritative state and never release an unsigned or unverified artifact.

Alert on age of the oldest queued job, time spent in each state, repeated delivery count, and terminal failures. A queue-depth graph alone can look healthy while one old patient document is stuck. The runbook should start from a job ID and reach its input digest, policy version, transition history, artifact digest, and signature result without opening the document itself.

The decision rule is compact: return a job ID once durable acceptance is complete, and declare success only after every release gate is recorded. If a shared backend contract lowers the full operating bill for your workload, start with the Infrai documentation and inspect the discovered schemas before implementing the adapter.

DEV Community: CarterHughes6849

Next.js API Error Tracking: Why I Capture Server Action Stack Traces

How should Next.js API routes capture server action errors?

My smallest useful capture boundary

Setup friction versus specialist depth

Reconstruct the incident, not just the exception

When I would choose another path

Sources

SMS OTP 2FA for B2B Approval Reminders — Suppression List Design

How should an SMS OTP 2FA suppression list block numbers?

Model the states before writing the handler

A small Go state machine for the safe path

Where do the provider choices differ?

Verify the rollout, then make rollback boring

References

Secure SMS OTP Login Flow Design: 5 Controls for Automotive Contact Routing

How should a secure SMS OTP login flow handle retry and lockout?

Five boundaries, five different failure modes

Make retries boring

Instrument the invariant, not the provider

Roll out without teaching production the lesson

Further reading

How to Build 2FA Login API in 2026: SMS OTP Resend Guardrails

How should a 2FA login API handle SMS OTP resend?

Implement the narrow HTTP boundary

Compare providers at the actual boundary

Verify before traffic, then observe the right evidence

Roll back without reopening old challenges

References

Image Deletion vs Video Deletion — Preventing Cross-Type Cleanup Errors

Two architectures, one invariant

How should image and video deletion prevent cross-type cleanup errors?

What do direct media services change in the decision?

Where this recommendation does not fit

References

Replaceable Image Moderation Pipelines — Durable Batch Jobs for Seller Catalogs

The signal that deserves a job record

How should seller catalog imports expose batch progress?

What changes when the supplier changes?

Verification and rollback

References

Hosted KPI Dashboard API in Node.js — Attributing AI Agent Batch Costs

Should a hosted KPI dashboard use a batch backend API?

Close the ledger before sending the batch

Compare destinations by the work left outside them

Verify failure behavior before trusting the chart

Rollback is a ledger operation

References

Storefront Hero Imagery: Go Saliency Crops for Desktop and Mobile Slots

What should a crop pipeline guarantee for desktop and mobile slots?

How do you implement deterministic smart crops in Go?

Where do bandwidth and visual quality trade off?

How do you verify, deploy, and roll back crop changes?

References

Generated Video Control with Go — Asynchronous Job Model, Cost, Capability Checks

Why is video generation an asynchronous job?

Choose the provider boundary before the worker

Implement one submission and many observations

Put storage and cache cost in the state machine

Verify, alert, and roll back

References

Video Generation Job Never Completes — Debug Status Polling and Cancel Timeouts

How do you debug status polling when a video generation job never completes?

What changes between the alert and cancellation?

Which service actually owns the stalled work?

What if the threshold is wrong?

Further reading

References

How to Handle PDF Decryption Failures — Supplier Password Errors in Production

What should the page say when PDF decrypt fails with a wrong password?

Implement the failure decision before adding retries

Instrument the signal that should fire first

Keep the decrypt provider replaceable

Close the incident without teaching the wrong lesson

Further reading

PDF Generation Queue: How Upload Endpoints Return Auditable Job IDs

How should an upload endpoint queue PDF generation?

Build the acceptance boundary

Make redaction and signature gates, not labels

Compare the operating bill, not one API call