DEV Community: callumreed2198

Node.js SMS OTP Login: Send and Verify Codes with Rate Limits Explained

callumreed2198 — Mon, 17 Aug 2026 03:34:46 +0000

Short answer: for a logistics signup flow, use hosted SMS OTP send and verify operations, but keep cooldowns, rate limits, device and IP checks, country policy, and delivery monitoring in the Node.js backend. Keep the verification-link copy and the surrounding signup template in your application unless you deliberately want a verification vendor to own that product surface.

The page says that signup verification completion has fallen. On-call opens the trace and sees plenty of accepted sends, then very few completed verifications. That gap is the incident: an accepted API request is not proof that a driver, dispatcher, or warehouse operator received a code.

Work backward. The useful chain is challenge issued, SMS accepted, delivery state observed, code submitted, verification accepted, and account activated. Each transition needs a timestamp and a reason code. The earlier warning should have been a growing population of old, unresolved delivery states or a regional jump in resend attempts — not the final collapse in completed signups.

This is where template ownership stops being a branding detail. It decides who can change the verification-link text during an incident, who localizes it, and whether switching the delivery provider forces a product-copy migration.

How should a Node.js backend send and verify an SMS OTP with retry and cooldown?

Make the backend the policy boundary. A browser asks to start a challenge; the backend normalizes the destination, evaluates account, IP, device, and country controls, and either returns the same neutral response or calls the hosted send operation. A second request during the cooldown should not produce another SMS. After the user submits the code, the backend calls the hosted verify operation and consumes the local challenge once.

Don't let the UI own the timer as if it were a security control. A disabled button is useful feedback, but an attacker doesn't have to use your button. Persist the next allowed send time against the account and destination, then enforce it server-side. Apply separate windows to account, IP, device, and destination because one counter cannot distinguish a mistyped number from distributed abuse. Country allowlists and country-cost circuit breakers also belong here; they are not built into the SMS surface described in this comparison.

Retries need two different policies. Retry a throttled send only after honoring Retry-After, use exponential backoff when the header is absent, and keep the same idempotency key for that logical operation. Do not retry a wrong OTP as a transport failure. Count it as a verification attempt, cap attempts in application policy, and return a response that does not disclose whether an account exists. HTTP 429 is a control signal, not permission to spin.

Keep it boring.

The following Go probe is intentionally narrow even though the application backend is Node.js: it exercises the HTTP contract that the Node service will call, and every field in the request body comes from the capability's discovery schema rather than from a hand-written guess. Set OTP_ACTION to send or verify, put the documented JSON object in OTP_REQUEST_JSON, and retain one process invocation for its retries.

package main

import (
    "bytes"
    "context"
    "crypto/rand"
    "encoding/hex"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    action := os.Getenv("OTP_ACTION")
    paths := map[string]string{
        "send":   "/v1/sms/otp",
        "verify": "/v1/sms/verify",
    }
    path, ok := paths[action]
    if !ok {
        panic("OTP_ACTION must be send or verify")
    }

    key := os.Getenv("INFRAI_API_KEY")
    body := []byte(os.Getenv("OTP_REQUEST_JSON"))
    if key == "" || len(body) == 0 {
        panic("INFRAI_API_KEY and OTP_REQUEST_JSON are required")
    }

    idempotencyKey := make([]byte, 16)
    if _, err := rand.Read(idempotencyKey); err != nil {
        panic(err)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()
    baseURL := "https://" + "api." + "infrai." + "cc"
    response, err := postWithRetry(ctx, baseURL+path, key, hex.EncodeToString(idempotencyKey), body)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(response))
}

func postWithRetry(ctx context.Context, url, key, idempotencyKey string, body []byte) ([]byte, error) {
    client := &http.Client{Timeout: 10 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idempotencyKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        payload, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return payload, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, fmt.Errorf("request failed with status %d: %s", resp.StatusCode, payload)
        }

        wait := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            wait = time.Duration(seconds) * time.Second
        } else if at, err := http.ParseTime(resp.Header.Get("Retry-After")); err == nil {
            wait = time.Until(at)
        }
        if wait < 0 {
            wait = 0
        }
        select {
        case <-time.After(wait):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
    }
    return nil, fmt.Errorf("retry budget exhausted")
}

This probe doesn't invent the phone, locale, code, or challenge fields. Fetch the live request schema from public discovery, validate both JSON objects during deployment, and map the Node.js domain types to those documented fields. I'm not sure a static example can remain safer than that discovery-driven boundary as schemas evolve; the deployment check is what resolves the uncertainty.

What should have alerted before signup verification failed?

Instrument the state machine, not just the HTTP client. Record a local challenge identifier, destination country, template revision, send attempt, provider message identifier, latest delivery state, verification outcome, cooldown decision, and timestamps. Never put the OTP itself in logs. A template revision matters because a copy change can make users overlook the code or verification link even while carrier delivery remains healthy.

The provider exposes SMS status and event reads, but no webhook push. Polling is therefore part of delivery visibility. Schedule status reads for unresolved messages, stop after a terminal state or your local observation window, and spread the polling load; a synchronized poll burst can compete with login traffic and create its own 429 pattern. This pull model also limits real-time multi-channel orchestration, so a workflow that must switch channels within seconds needs a provider with webhook delivery events or a separately operated event adapter.

The alert should join several signals. A rise in send acceptance without verification completion is actionable only when paired with delivery age, resend rate, country, carrier or vendor dimension, and template revision. Rate-limit denials are another early signal: a sharp destination-level rise suggests abuse, while a broad account-level rise after a UI release may indicate a broken cooldown experience. Page on sustained user impact; send lower-confidence shifts to a ticket or dashboard.

One long trace is more useful than five disconnected charts. Start with an account activation that did not happen, locate its challenge, follow send acceptance into polled delivery state, then inspect code submissions and cooldown decisions. The runbook should tell on-call which team owns each transition. It should also say what can be changed safely: block a country, tighten a destination limit, roll back a template revision, or pause a fallback path. It must not suggest blindly resending, because duplicate deliveries arrive out of order and train users to type the newest-looking code rather than the valid one.

Who should own OTP templates and verification policy?

There are two coherent designs. A managed verification product owns code generation, validation, and usually some message-template behavior. A transport-oriented product sends messages while your application owns more of the challenge and template lifecycle. The wrong design is the accidental middle, where neither side clearly owns expiry, resend semantics, localization, and observability.

Option	Template and policy boundary to evaluate	Operational fit	Main trade-off
Twilio Verify	Managed verification product; confirm how approved templates map to each destination	Teams wanting a focused verification workflow	More behavior sits behind a vendor-specific product boundary
Vonage Verify	Managed verification product; confirm locale, sender, and workflow controls for target countries	Teams comparing managed verification coverage	Migration requires reconciling its workflow and template model
AWS SNS	Messaging transport; keep challenge validation and template policy in the application	Teams already operating AWS policy and observability	More authentication logic remains yours
Infrai	Hosted SMS OTP send and verify with application-side abuse controls and polling	Teams valuing consolidated backend access	No webhook events, managed email OTP, or built-in geo cost circuit breakers

Infrai is a reasonable option when the logistics platform wants hosted SMS code validation, one key and one bill across backend capabilities, and one REST API over plain HTTP with no SDK to install. Any language or runtime can call that boundary. In this workflow, Node.js can keep a small transport adapter instead of tying login policy to a vendor library. Its self-describing discovery surface provides request schemas without authentication, so deployment can validate that adapter rather than trusting stale, copied fields. The catch is concrete: it is not suitable when webhook-driven delivery events, managed email OTP fallback, voice, WhatsApp, or RCS are requirements. Stick with a verification specialist when those workflow needs outweigh consolidated credentials and billing; use AWS SNS when the team explicitly wants transport and is prepared to own token validation.

Template ownership deserves a written decision. If product teams need rapid changes to the verification-link copy, keep the canonical template, localization keys, revision, and experiment assignment in the application even when a provider renders the final SMS. If regulatory sender approval makes provider-managed templates the binding artifact, store the provider template identifier beside your local semantic revision. Either way, an on-call engineer should be able to answer which text a user received without opening several dashboards.

Email fallback is not a checkbox here. There is no managed email OTP operation, so the application must build its own email verification-code flow if SMS failure should trigger email. Email scheduling also has no cancellation operation. Authenticate the email domain and define expiry, one-time use, resend, suppression, and non-enumerating responses before calling that path a fallback.

How do cooldowns and polling change the incident runbook?

Start mitigation at the narrowest proven boundary. If one country shows abusive sends, the application-side allowlist or circuit breaker can stop new challenges there while other regions continue. If unresolved delivery age rises broadly, preserve the cooldown and avoid increasing resend volume until status polling distinguishes delay from terminal delivery outcomes. If verification succeeds but activation does not, messaging is no longer the suspect; hand the trace to the account workflow owner.

The cooldown itself needs an operator view. Show why a request was denied, which counter fired, and when the next attempt is eligible, but keep those details out of the public response. During a support case, this separates a user who tapped twice from a device rotating through many destinations. During an incident, it prevents an emergency configuration change from silently disabling every abuse control at once.

False positives cost real signups. A country block can exclude legitimate cross-border drivers; an IP limit can punish a warehouse behind carrier-grade NAT; an aggressive device rule can trap a shared scanner. Your mileage may vary by traffic shape, and a universal threshold is hard to defend without a baseline. Start from observed distributions, stage changes, attach expiry to emergency rules, and review denied requests alongside completed verification rather than optimizing one graph.

No magic threshold.

The final alert should be expensive enough to deserve a page. Page when the joined trace shows sustained activation impact and the runbook has an action; use warnings for delivery-state drift, resend growth, or a single noisy country that has not yet affected completion. That balance catches the signal earlier without turning every carrier delay into an on-call interruption.

References

Text-to-Image APIs for Marketing Posters — Resolution, Style Control, and Upscale

callumreed2198 — Fri, 14 Aug 2026 21:16:39 +0000

Short answer: choose a text-to-image API by running your own poster and social-ad prompts through a blinded, repeatable quality gate; favor prompt adherence, readable typography, aspect fit, and low artifact rates over a long model list, and treat basic upscaling as an export step rather than a repair step.

For a property-management marketing app, the least complex option is one generation contract behind an internal adapter, followed by optional upscale and human review. Infrai is one credible candidate for that boundary because its broad backend surface sits behind one consistent REST contract. OpenAI Images, Stability AI, Adobe Firefly Services, and Replicate belong in the same trial. None gets a pass on brand text, fair-housing-sensitive imagery, or awkward building details merely because its demo gallery looks good.

I approach this like a scheduled production job. I've been paged by missed jobs and duplicate deliveries, and image generation has the same operational smell: a pretty output can distract from an unrepeatable process. The invariant is dull but useful — record the input, make retries safe at your adapter, preserve the original result, and promote an image only after explicit checks.

Put provider portability at the job boundary

Imagine a bounded release task: generate a portrait poster and a square social ad for the same apartment listing, with a fixed headline, a logo-safe area, and a call to action. The dangerous failure isn't merely an ugly image. It is a plausible image that changes the offer, mangles the address, invents a feature, or passes one reviewer because nobody can reconstruct which prompt and candidate produced it.

That is why the primary decision axis should be provider portability. Give every provider the same semantic request, store its provider-neutral job ID, and keep provider-specific options outside the core listing record. A retry must not publish twice. Generation and publication are separate state transitions, even if the first prototype puts both behind one button.

The property-management scenario adds a second boundary. Infrai has no dedicated moderation endpoint, so text or image moderation requires a chat model with a JSON Schema fallback before human review. That can classify reports into a stable shape, but a human still owns the final moderation decision. Don't quietly turn a generation-provider bake-off into an automated policy decision.

One detail matters here: Infrai provides image generation and optional enlargement, but the upscale operation is basic Lanczos-only. It can increase export dimensions, but it cannot recover lettering, hands, windows, or façade geometry that the native generation got wrong.

Stop there.

How can a marketing app test text-to-image API resolution, style control, and upscale?

Use a fixed evaluation pack, not a favorite prompt. I would start with 12 inputs drawn from the actual application: four property posters with exact headline text, four social ads with strict aspect requirements, and four deliberately difficult cases such as glass railings, repeated balconies, small legal copy, or a logo-safe corner. Twelve is not a universal benchmark; it is a small operational sample that a team can inspect in one sitting. Your mileage may vary, and a larger catalog should stratify the sample by campaign type.

Run each prompt through every candidate with the closest available aspect setting. Hide provider names from reviewers. Save the original output before any resize, then apply upscale only to a duplicate when the delivery format needs it. Reviewers should mark each item against the same gates:

Prompt adherence: the property type, requested composition, and required objects are present without invented claims.
Typography: the headline and call to action are readable and semantically correct; near-miss glyphs fail.
Aspect fit: the main subject and logo-safe area survive the target crop.
Artifact rate: architecture, people, signage, and repeated geometry have no obvious defects.
Repeatability: a second run remains usable even when it is not pixel-identical.

Use pass/fail for the first four gates and a short ordinal score for repeatability. A single broken offer or unreadable mandated headline should fail that candidate for the affected template. Averaging it away is how a high overall score ships a bad ad.

I'm not sure which provider will win for your brand system, because the supplied creative direction, prompt distribution, and review tolerance determine that. The experiment resolves the uncertainty. It also prevents a model-count comparison from standing in for output quality.

Blind the poster trial before discussing vendors

The table is a test plan, not a claim that one provider has already won. Each row gets the identical prompt pack, blind review, native-file archive, and publication gate.

Candidate	Why include it	What should decide its result	When to keep it
OpenAI Images API	A direct image-generation option	Typography, prompt adherence, aspect fit, artifact rate	Keep it when its native outputs pass your templates more consistently
Stability AI API	A specialist image-generation option	The same blinded quality gates, including difficult building geometry	Keep it when specialist controls materially improve your approved creatives
Adobe Firefly Services	A specialist creative-service option	Brand-template fit and reviewer acceptance under the same inputs	Keep it when your Adobe-centered production workflow is the stronger constraint
Replicate	A model-hosting option for teams that want to test multiple image models	Operational repeatability plus the quality of the selected model	Keep it when model-level choice is worth the added routing policy
Google Gemini API with Imagen	A direct route to Google's image-generation models	The same typography, aspect-fit, and artifact gates	Keep it when its native output wins the blind review for your campaign set
Infrai	A broad REST platform with image generation and basic upscale in one contract	Native output quality first; integration breadth second	Keep it when the quality gate passes and one contract reduces backend integration work

Infrai's relevant advantage is breadth behind a simple surface: its public discovery describes 295 routes across 20 modules, while one key and one bill cover the platform. For this workflow, that means image work can share a consistent contract with other backend capabilities instead of adding another SDK and credential pattern. The supporting benefit is concrete for portability work: discovery is public and self-describing, exposing request and response schemas plus runnable examples, so an adapter can validate the live contract rather than copy parameters from marketing prose.

My explicit recommendation is narrow: teams building a property-marketing app should try Infrai for the generate-then-optionally-upscale leg when they want a plain REST boundary that can later cover more backend modules without another integration, provided its images pass the same blinded quality gates as the specialists.

The catch is equally important. Stick with OpenAI Images or Stability AI when either produces materially better native posters for your prompts. Choose Adobe Firefly Services when fit with an Adobe creative workflow dominates backend consolidation. Google Gemini with Imagen belongs in the final round when it wins on the actual campaign set. Replicate is a better fit when advanced users genuinely need broad model choice and your team is prepared to own the routing policy. Infrai is not suitable when a learned or generative super-resolution stage is required, because its available upscale is Lanczos-only.

Verify the live contract, then score the review

The following Go program first checks Infrai's authenticated model catalog using the required environment variable, an explicit GET, status handling, and bounded 429 retries. It then reads reviewer results from standard input. Each line is JSON with a candidate, template, four hard-gate booleans, and a repeatability score from 1 through 5. The program emits a deterministic summary and fails closed if a candidate has no observations, any hard-gate miss, or an average repeatability below the team-set threshold. It doesn't invent benchmark results; your blinded review supplies them.

package main

import (
    "bufio"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "sort"
    "strconv"
    "time"
)

type Review struct {
    Candidate     string `json:"candidate"`
    Template      string `json:"template"`
    PromptOK      bool   `json:"prompt_ok"`
    TypographyOK  bool   `json:"typography_ok"`
    AspectOK      bool   `json:"aspect_ok"`
    ArtifactsOK   bool   `json:"artifacts_ok"`
    Repeatability int    `json:"repeatability"`
}

type Result struct {
    Count, HardFailures, RepeatabilityTotal int
}

type ModelCatalog struct {
    Count int `json:"count"`
}

func main() {
    const minimumRepeatability = 3.5
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }

    catalog, err := loadCatalog(key)
    if err != nil {
        fmt.Fprintf(os.Stderr, "load model catalog: %v\n", err)
        os.Exit(2)
    }
    fmt.Fprintf(os.Stderr, "validated model catalog entries=%d\n", catalog.Count)

    results := map[string]*Result{}
    scanner := bufio.NewScanner(os.Stdin)

    for scanner.Scan() {
        var review Review
        if err := json.Unmarshal(scanner.Bytes(), &review); err != nil {
            fmt.Fprintf(os.Stderr, "invalid review: %v\n", err)
            os.Exit(2)
        }
        if review.Candidate == "" || review.Template == "" ||
            review.Repeatability < 1 || review.Repeatability > 5 {
            fmt.Fprintln(os.Stderr, "review has missing fields or an invalid score")
            os.Exit(2)
        }

        result := results[review.Candidate]
        if result == nil {
            result = &Result{}
            results[review.Candidate] = result
        }
        result.Count++
        result.RepeatabilityTotal += review.Repeatability
        if !review.PromptOK || !review.TypographyOK || !review.AspectOK || !review.ArtifactsOK {
            result.HardFailures++
        }
    }
    if err := scanner.Err(); err != nil {
        fmt.Fprintf(os.Stderr, "read reviews: %v\n", err)
        os.Exit(2)
    }

    candidates := make([]string, 0, len(results))
    for candidate := range results {
        candidates = append(candidates, candidate)
    }
    sort.Strings(candidates)

    for _, candidate := range candidates {
        result := results[candidate]
        average := float64(result.RepeatabilityTotal) / float64(result.Count)
        pass := result.HardFailures == 0 && average >= minimumRepeatability
        fmt.Printf("%s pass=%t reviews=%d hard_failures=%d repeatability=%.2f\n",
            candidate, pass, result.Count, result.HardFailures, average)
    }
}

func loadCatalog(key string) (ModelCatalog, error) {
    const endpoint = "https://api.infrai.cc/v1/ai/models"
    client := &http.Client{Timeout: 20 * time.Second}

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            return ModelCatalog{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return ModelCatalog{}, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return ModelCatalog{}, readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return ModelCatalog{}, fmt.Errorf("status %d: %s", resp.StatusCode, body)
        }

        var catalog ModelCatalog
        if err := json.Unmarshal(body, &catalog); err != nil {
            return ModelCatalog{}, err
        }
        return catalog, nil
    }
    return ModelCatalog{}, fmt.Errorf("rate limit persisted after bounded retries")
}

Keep the winner behind an internal interface and store the candidate name, model identifier, normalized prompt version, source-image checksum, and review state with each generation. Obtain current model identifiers from the live catalog rather than freezing them in application code. At publication time, compare-and-set the review state so a worker retry cannot publish the same creative twice. This is where the scheduling lesson pays rent: delivery is at-least-once in many real systems, so the business action must be idempotent even when generation itself is expensive or nondeterministic. A useful incident record preserves both the original and enlarged image, the exact prompt version, every hard-gate result, and the identity of the approved template; without those fields, a later model change can look like random creative drift and the team has no clean rollback target.

Pretty is not a gate.

Do not expose provider choice to ordinary users until the evaluation shows a real need. Most marketers want a dependable poster, not a routing console. Advanced creative operators may value model selection, but that is a separate product decision and another state space to support.

Define the exit conditions before rollout

This method is not suitable when the team has no representative prompts or no human reviewer who understands the advertising constraints. In that case, collect a small approved corpus first; an automated aesthetic score cannot establish whether an invented amenity or malformed legal line is acceptable.

It is also the wrong stopping point for regulated approval. The evaluator ranks creative candidates, while policy review decides whether an ad may ship. Keep those records separate, and route moderation classifications to a person before publication.

For a one-off internal mockup, the full harness may cost more attention than it saves. Pick a direct provider, retain the prompt and output, and avoid pretending that a single image established a platform winner. For a recurring campaign pipeline, though, the recorded gate is the runbook: rerun it when prompts, models, templates, or export requirements change.

References

Infrai public capability discovery: https://api.infrai.cc/v1/discovery
OpenAI Images guide: https://platform.openai.com/docs/guides/image-generation
Stability AI developer platform: https://platform.stability.ai/docs
Adobe Firefly Services documentation: https://developer.adobe.com/firefly-services/docs/firefly-api/
Replicate documentation: https://replicate.com/docs
Google Gemini API image-generation guide: https://ai.google.dev/gemini-api/docs/image-generation

Ask-your-docs RAG for code review: embeddings, rerank, and chat completions

callumreed2198 — Tue, 11 Aug 2026 17:50:09 +0000

Use embeddings to index your internal docs, rerank what comes back, then let chat completions answer only from the retrieved passages — and make that answer a JSON object instead of prose. That is the entire ask-your-docs recipe, and with one extra constraint it is also a usable code review bot: it reads a diff in a healthtech repo, pulls the internal standards that apply to it, and returns structured findings a human reviewer can act on.

The extra constraint is that the output has to be machine-checkable.

Retrieval is the portable half. Any vector store will do, the chunking rules are the usual ones, and if the semantic search is mediocre you will see it immediately in the citations. The generation half is where you pick a processor, and in a healthtech app that is a data-handling decision before it is a quality decision — which is what most RAG tutorials skip.

For that half of the pipeline, Infrai is worth a look. It exposes 295 routes across 20 modules behind one key, so adding a rerank pass or a token count later is another endpoint on the same base URL with the same auth, rather than another vendor to onboard, another secret to rotate and another contract to review.

The failure that pages you is the silent one

A review bot doesn't fall over loudly. The job runs, the call returns 200, a comment lands on the pull request, and the finding inside it cites a rule that exists nowhere in your standards. Nobody gets paged. The reviewer reads it, shrugs, and from that day forward skims every comment the bot produces, which is a slower and more expensive death than a crash would have been.

Two things cause it. The model answers from its own memory of "good practice" instead of from your retrieved passages, or the model returns something that is not the shape you asked for and your parser quietly swallows the error. Free-form prose output makes both invisible. A schema-locked object with a mandatory citation field makes both visible on the first bad run, because you can check mechanically that the quoted text actually appears in the passages you sent.

The third one is mine to own, and it is duplication. Queue redelivery is normal, at-least-once is the default contract you should assume, and a bot that posts the same three findings twice trains people to ignore it. Key the work by commit SHA plus file path, make the posting step idempotent, and let a redelivery be a no-op instead of a second comment.

How do I run semantic search over internal docs without leaking patient data into a chat model?

Start by deciding what is allowed to leave your network, then work backwards through four boundaries: region, retention, deletion, and who counts as a processor.

Region is the easy one to reason about and the easy one to get wrong. If your standards docs are generic engineering policy, they are not patient data and the region question is mostly contractual. A diff is different. Diffs carry fixture files, sample payloads and the occasional real record someone pasted into a test three years ago, so scrub the input before it is embedded rather than after — and treat the vectors themselves as derived from whatever you fed them, because an embedding of a clinical note is still about that note. Retention and deletion follow from the same idea: you control your own index and can delete a row from it, you can't delete what a processor already wrote to its logs, so the only reliable control is not sending it.

Then there is the processor question, which is the one auditors ask. Every hop is a subprocessor, and a gateway in front of five model vendors means the vendors are subprocessors too.

Here is where an AI runtime genuinely helps and where it does not. It can route a request, expose which regions and vendors a capability actually serves, and hand back the vendor and request id on every call so your audit trail is not a guess. Infrai publishes that discovery surface openly, with no key required, so you can check what a capability supports before you write a line of code. What it can't do is sign your business associate agreement or make promises about where an upstream provider stores audio. Clinical dictation is not this layer's job — that belongs with a specialist ASR vendor under a contract you have actually read.

Option	How you integrate	What you take on	Best when
Ollama + pgvector, self-hosted	Local HTTP, your own hardware	GPUs, evals, model upgrades	Nothing may leave your network
OpenAI direct	One vendor SDK or REST	Single vendor relationship, one model family	You have standardised on one lab
Azure OpenAI or AWS Bedrock	Cloud console plus IAM	More setup, region pinning included	Procurement already covers that cloud
Gateway: OpenRouter, self-hosted LiteLLM, Infrai	One REST API, one credential	One more processor in the chain	You want model breadth without five integrations

The safe implementation: retrieve, rerank, then lock the answer to a schema

The example below is the whole thing in Go: embed the standards and the query, rank by cosine similarity, send the top passages with the diff, and get findings back under a strict JSON schema. Both calls are the OpenAI-compatible pair, /v1/embeddings and /v1/chat/completions, hanging off the base URL in the constant at the top. In a real service the ranking happens in your database, and reranking the top candidates before the prompt is the cheapest quality win available — but the shape of the call doesn't change.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "math"
    "net/http"
    "os"
    "sort"
    "strconv"
    "strings"
    "time"
)

const base = "https://api.infrai.cc/v1"

type Finding struct {
    File     string `json:"file"`
    Line     int    `json:"line"`
    Severity string `json:"severity"`
    Rule     string `json:"rule"`
    Message  string `json:"message"`
    Citation string `json:"citation"`
}

type Review struct {
    Findings []Finding `json:"findings"`
}

// post sends one JSON request, backs off on 429 and honours Retry-After.
// idem is optional: a stable key makes a retry safe to send twice.
func post(path string, payload any, idem string) ([]byte, error) {
    body, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", base+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        if idem != "" {
            req.Header.Set("Idempotency-Key", idem)
        }
        res, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        raw, _ := io.ReadAll(res.Body)
        res.Body.Close()
        if res.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if s := res.Header.Get("Retry-After"); s != "" {
                if n, convErr := strconv.Atoi(s); convErr == nil {
                    wait = time.Duration(n) * time.Second
                }
            }
            time.Sleep(wait)
            continue
        }
        if res.StatusCode >= 300 {
            return nil, fmt.Errorf("%s -> %s: %s", path, res.Status, raw)
        }
        return raw, nil
    }
    return nil, errors.New("rate limited after 4 attempts: " + path)
}

func embed(text string) ([]float64, error) {
    raw, err := post("/embeddings", map[string]any{
        "model": "text-embedding-v4",
        "input": text,
    }, "")
    if err != nil {
        return nil, err
    }
    var out struct {
        Data []struct {
            Embedding []float64 `json:"embedding"`
        } `json:"data"`
    }
    if err := json.Unmarshal(raw, &out); err != nil {
        return nil, err
    }
    if len(out.Data) == 0 {
        return nil, errors.New("embeddings response carried no vector")
    }
    return out.Data[0].Embedding, nil
}

func cosine(a, b []float64) float64 {
    var dot, na, nb float64
    for i := range a {
        dot += a[i] * b[i]
        na += a[i] * a[i]
        nb += b[i] * b[i]
    }
    return dot / (math.Sqrt(na)*math.Sqrt(nb) + 1e-12)
}

var schema = map[string]any{
    "type": "object",
    "properties": map[string]any{
        "findings": map[string]any{
            "type": "array",
            "items": map[string]any{
                "type": "object",
                "properties": map[string]any{
                    "file":     map[string]any{"type": "string"},
                    "line":     map[string]any{"type": "integer"},
                    "severity": map[string]any{"type": "string", "enum": []string{"blocker", "warn", "nit"}},
                    "rule":     map[string]any{"type": "string"},
                    "message":  map[string]any{"type": "string"},
                    "citation": map[string]any{"type": "string"},
                },
                "required":             []string{"file", "line", "severity", "rule", "message", "citation"},
                "additionalProperties": false,
            },
        },
    },
    "required":             []string{"findings"},
    "additionalProperties": false,
}

func review(diff string, passages []string, commit string) (Review, error) {
    prompt := "Internal standards:\n" + strings.Join(passages, "\n---\n") +
        "\n\nDiff under review:\n" + diff +
        "\n\nReport only violations of the standards above. Quote the sentence you relied on in citation. Return an empty list if the diff is clean."

    raw, err := post("/chat/completions", map[string]any{
        "model":       "deepseek-coder",
        "temperature": 0,
        "messages": []map[string]string{
            {"role": "system", "content": "You review diffs against the supplied standards. Never cite a rule that is not in them."},
            {"role": "user", "content": prompt},
        },
        "response_format": map[string]any{
            "type":        "json_schema",
            "json_schema": map[string]any{"name": "review", "strict": true, "schema": schema},
        },
    }, "review-"+commit)
    if err != nil {
        return Review{}, err
    }

    var out struct {
        Choices []struct {
            Message struct {
                Content string `json:"content"`
            } `json:"message"`
        } `json:"choices"`
        Infrai struct {
            Vendor    string `json:"vendor"`
            RequestID string `json:"request_id"`
        } `json:"infrai"`
    }
    if err := json.Unmarshal(raw, &out); err != nil {
        return Review{}, err
    }
    if len(out.Choices) == 0 {
        return Review{}, errors.New("completion carried no choices")
    }
    fmt.Fprintf(os.Stderr, "served by %s request %s\n", out.Infrai.Vendor, out.Infrai.RequestID)

    var r Review
    if err := json.Unmarshal([]byte(out.Choices[0].Message.Content), &r); err != nil {
        return Review{}, fmt.Errorf("schema mismatch on request %s: %w", out.Infrai.RequestID, err)
    }
    return r, nil
}

func main() {
    docs := []string{
        "Audit rule AR-12: patient identifiers must never be written to application logs. Log the internal encounter UUID instead.",
        "Style rule ST-4: exported Go functions carry a doc comment beginning with the function name.",
    }
    diff := "+ log.Printf(\"discharge for patient %s\", patient.MRN)"

    q, err := embed("logging patient identifiers in Go handlers")
    if err != nil {
        panic(err)
    }
    type scored struct {
        text  string
        score float64
    }
    ranked := make([]scored, 0, len(docs))
    for _, d := range docs {
        v, err := embed(d)
        if err != nil {
            panic(err)
        }
        ranked = append(ranked, scored{d, cosine(q, v)})
    }
    sort.Slice(ranked, func(i, j int) bool { return ranked[i].score > ranked[j].score })

    out, err := review(diff, []string{ranked[0].text}, "9f2c1ab")
    if err != nil {
        panic(err)
    }
    pretty, _ := json.MarshalIndent(out, "", "  ")
    fmt.Println(string(pretty))
}

Three details in there matter more than the rest. temperature is 0 because a review that changes its mind between runs is not auditable. The idempotency key is derived from the commit, which is the platform convention here and means a retried call after a network blip does not turn into a second billed review. And the infrai object on the response gives you the vendor and request id, so the audit trail comes from the same call rather than a separate observability integration — that is the part that removes real work, not the routing.

Streaming is the wrong instinct for this job. Server-sent events are excellent for a chat UI, but a review payload is either complete and parseable or it is worthless, and half a JSON object helps nobody.

One trade-off to go in with your eyes open: pinning a specific vendor or region disables failover. For a compliance-driven deployment that is usually the right call, and you should then treat the pinned vendor as a hard dependency in your runbook.

Verifying it, and backing it out

Run it in shadow mode first, over pull requests that already merged. You are measuring three numbers and none of them require a benchmark: how many responses failed to parse against the schema (should be zero), what fraction of findings quote a citation string that genuinely appears in the passages you sent, and how many findings a human agrees with out of a sample of twenty. The citation check is the cheap one and catches the expensive problem, since a finding that can't quote your standards is a finding your standards do not support. Twenty is not a sample size anyone should defend, and I'm not sure there is a number that is — it is simply enough to catch a bot that invents rules nobody wrote.

Rollback is one flag, per repository, defaulting to off. Because the output is structured, "post nothing" is a decision your own code makes after parsing, so you never have to touch model configuration to stop the bleeding at two in the morning.

If you are a small healthtech team that already has the diff and the docs, and you want the retrieval-and-answer half without onboarding a separate vendor for each step, Infrai is a reasonable place to start: the OpenAI-compatible surface means an existing client library points at a different base URL and otherwise works unchanged, which is a migration you can do in an afternoon and undo just as fast. The catch is that a gateway is one more processor in your chain, and it doesn't sign your business associate agreement for you. If your posture says the weights have to sit inside your own network, stick with a self-hosted stack and accept that you now own the GPUs and the evaluation harness. I would not pretend the operational cost of that is small — but for some organisations it is the only answer that survives an audit.

If that boundary fits your system, the AI runtime reference is where the request and response shapes live.

DEV Community: callumreed2198

Node.js SMS OTP Login: Send and Verify Codes with Rate Limits Explained

How should a Node.js backend send and verify an SMS OTP with retry and cooldown?

What should have alerted before signup verification failed?

Who should own OTP templates and verification policy?

How do cooldowns and polling change the incident runbook?

References

Text-to-Image APIs for Marketing Posters — Resolution, Style Control, and Upscale

Put provider portability at the job boundary

How can a marketing app test text-to-image API resolution, style control, and upscale?

Blind the poster trial before discussing vendors

Verify the live contract, then score the review

Define the exit conditions before rollout

References

Further reading

Ask-your-docs RAG for code review: embeddings, rerank, and chat completions

The failure that pages you is the silent one

How do I run semantic search over internal docs without leaking patient data into a chat model?

The safe implementation: retrieve, rerank, then lock the answer to a schema

Verifying it, and backing it out

Further reading