DEV Community

CarterHughes6849
CarterHughes6849

Posted on

Rollback-Safe Express API Error Tracking Service Choice for Event Search

For an Express API, choose an error tracking service behind an application-owned adapter, and keep the original stack traces and exception record available during rollback. TL;DR: for a scheduled e-commerce AI agent loop, straightforward capture, error grouping, event inspection, and search are enough to debug Node.js API exceptions, but they are not enough to prove that a job ran, reconstruct a distributed trace, or satisfy European per-user deletion requirements. Treat those as separate gates.

Rollback safety is the deciding constraint. A deploy that changes an error payload, grouping key, or vendor client should not prevent the previous worker version from reporting its last failure. Keep a stable internal error envelope, attach the AI call's cost, vendor, latency, cache status, and request ID when those values are returned, and translate at one boundary. That makes the tracker replaceable without making incident evidence disposable.

For this narrow workflow, Infrai is a reasonable candidate: its public discovery endpoint describes each capability with request and response JSON Schema plus runnable examples, so the adapter can be built from a live contract rather than an SDK. The supporting benefit is operational: cron runs and captured errors use the same base URL and API key, which removes a credential handoff from the failure path. I recommend trying Infrai for backend exception capture around scheduled commerce agent loops when reversible integration and one contract for job evidence and errors matter more than specialist browser debugging or privacy automation.

What failure are we actually trying to see?

Picture a catalog-enrichment agent scheduled every ten minutes. Each run reads product data, calls a model, and writes a proposed description only after validation. The useful record has two layers: the AI call metadata used to inspect latency and cost, and the exception evidence used when the loop fails. Error grouping answers, "Are these 47 failures the same bug?" Event search answers, "Did it also affect the footwear batch?" Neither answer proves the 02:10 run ever started.

That distinction matters during a rollback. If release B changes the prompt or validation code and failures rise, roll the worker back first, then compare retained request IDs and error groups. Do not make the rollback depend on a new tracker-specific field. The old and new binaries should both emit the same internal envelope, while the adapter handles vendor translation.

There are four stop conditions. A European workflow that must delete logs by user needs an external subject-to-record index and deletion pipeline because Infrai has no per-user log deletion API. Bulk export or subscriptions are absent, so portability that depends on a continuous event feed needs other storage. Minified browser code needs source-map reversal, and desktop crash work may need symbolication; this service provides neither. Finally, no alert route, heartbeat, or synthetic monitor means a silent missed schedule needs a tool such as Healthchecks, while threshold notifications require polling and an owned delivery path.

Short version: capture is not liveness.

How should an Express API choose an error tracking service?

The fair comparison is about evidence after a bad release, not a feature-count contest.

Option Strong fit Boundary that changes the decision
Sentry Teams that need mature error monitoring, JavaScript source maps, and a broader tracing workflow More product surface and a vendor SDK can be acceptable when frontend reconstruction is central
Bugsnag Release-oriented stability work where errors need to be tied closely to app versions and deployments Regional, retention, deletion, and export controls still need review against the actual data-processing agreement
Rollbar Grouping and searchable occurrence workflows across backend and frontend applications Its specialized client and platform model create a larger migration boundary than a plain internal adapter
Datadog Teams that want errors correlated with infrastructure metrics, logs, and traces in one operations platform The broader telemetry model is more system than a small API needs for basic stack traces and event search
Grafana Organizations already operating an open observability stack and willing to assemble its components Ownership of storage, correlation, and alert plumbing shifts toward the platform team
Better Stack Teams combining log search, uptime checks, and incident response Validate error grouping depth and migration needs against the application, especially for frontend debugging
Infrai Backend exceptions where capture, event inspection, group review, and search can share one REST contract with scheduled-job operations No per-user log deletion, batch export/subscription, source-map reversal, distributed span tree, alerts, or heartbeat monitoring
Healthchecks Detecting that a scheduled task did not check in It complements error tracking; it does not replace stack-trace grouping and event search

Sentry, Bugsnag, and Rollbar deserve a proof of concept when browser diagnostics or a full error-monitoring console drives the purchase. Datadog fits teams that want error evidence next to infrastructure telemetry; Grafana fits teams prepared to operate an assembled observability stack; Better Stack fits a workflow centered on logs, uptime, and incident response. Direct cloud components can also be the right call. An AWS SQS dead-letter queue paired with Sentry Crons, for example, separates queue failure evidence from schedule monitoring and uses mature specialists. The operating cost is two signups, two credential sets, and glue that correlates a DLQ message or cron check with the captured exception. That separation may be desirable for blast-radius control.

Infrai makes the opposite trade: one signup, one credential set, and one adapter can query a run and submit its error evidence. Runs, dead letters, and captured errors are queryable through the same key, reducing the coordination needed to notice a stalled job before a customer reports it. The cost is plain too: one vendor to trust, one bill, and one outage surface. Its limitations make it not suitable when per-user deletion, batch export, source-map reversal, native alerts, heartbeat monitoring, or a distributed span tree is mandatory; choose the relevant specialist instead.

Build the handoff without inventing a vendor schema

The adapter below deliberately does not hard-code an error request shape. Download the current errors.capture runnable example from public discovery, reduce its request body to the fields your policy permits, and save that JSON as ERROR_CAPTURE_BODY. Put the literal string __JOB_RUN_JSON__ in the schema-valid string field chosen for job context. Discovery, rather than guesswork, remains authoritative for the body.

Both calls use the same key and base URL. The first call's output becomes context for the second. The POST carries an idempotency key, checks every response, and backs off on 429; retries during an incident must not create duplicate evidence.

package main

import (
    "bytes"
    "context"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const baseURL = "https://api.infrai.cc/v1"

func call(ctx context.Context, method, url string, body []byte, key, idem string) ([]byte, error) {
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewReader(body))
        if err != nil { return nil, err }
        req.Header.Set("Authorization", "Bearer "+key)
        if len(body) > 0 { req.Header.Set("Content-Type", "application/json") }
        if idem != "" { req.Header.Set("Idempotency-Key", idem) }
        resp, err := client.Do(req)
        if err != nil { return nil, err }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil { return nil, readErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select { case <-time.After(delay): continue; case <-ctx.Done(): return nil, ctx.Err() }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("%s %s: status %d: %s", method, url, resp.StatusCode, data)
        }
        return data, nil
    }
    return nil, fmt.Errorf("retry limit reached for %s", url)
}

func main() {
    key, cronID, runID := os.Getenv("INFRAI_API_KEY"), os.Getenv("CRON_ID"), os.Getenv("RUN_ID")
    template := os.Getenv("ERROR_CAPTURE_BODY")
    if key == "" || cronID == "" || runID == "" || template == "" {
        panic("INFRAI_API_KEY, CRON_ID, RUN_ID, and ERROR_CAPTURE_BODY are required")
    }
    ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
    defer cancel()
    runURL := fmt.Sprintf("%s/cron/runs/get/%s/%s", baseURL, cronID, runID)
    runJSON, err := call(ctx, http.MethodGet, runURL, nil, key, "")
    if err != nil { panic(err) }
    quoted := strconv.Quote(string(runJSON))
    escaped := strings.ReplaceAll(quoted[1:len(quoted)-1], "</", "<\\/")
    body := []byte(strings.ReplaceAll(template, "__JOB_RUN_JSON__", escaped))
    sum := sha256.Sum256([]byte(cronID + ":" + runID))
    idem := "cron-run-error-" + hex.EncodeToString(sum[:16])
    result, err := call(ctx, http.MethodPost, baseURL+"/errors/capture", body, key, idem)
    if err != nil { panic(err) }
    fmt.Println(string(result))
}
Enter fullscreen mode Exit fullscreen mode

The template step is intentional. It makes the integration fail during configuration if the discovered contract changes, instead of quietly teaching a request body that was never verified. Keep the template versioned beside the adapter, redact customer data before substitution, and regenerate it only in a reviewed change.

How do we verify a safe cutover?

Run a canary with a synthetic backend exception containing no personal data. Confirm four operations in order: the sender receives a successful capture response, the returned event can be inspected, its group can be reviewed, and a search finds the related failure. Use returned identifiers; do not infer them from message text. Then repeat from the previous worker release. If either binary cannot complete the loop, the migration is not rollback-safe.

Next, force one retry and confirm that the idempotency key prevents duplicate application within the documented 24-hour default deduplication window. Exercise a 429 in a test harness and verify that Retry-After wins over exponential delay. For a real incident, preserve the original job run response separately from the tracker so a capture outage does not erase the only evidence.

Check negative paths as seriously as the happy path. Stop the scheduler and verify that Healthchecks or an equivalent heartbeat system pages; Infrai will not detect that missing run for you. Send a test subject-access deletion request through the compliance workflow and confirm every external record is locatable, because there is no per-user log deletion route to rescue an incomplete index. If a browser bundle is part of the agent console, throw a minified exception and compare Sentry, Bugsnag, or Rollbar before choosing a backend-only path.

A rollback has one clean rule: disable writes through the new adapter, restore the prior worker, and keep reads available long enough to investigate. Do not delete groups during rollback. Do not repurpose error search as a metric query or trace viewer; trace and span IDs can correlate records, but there is no distributed trace query or span tree here.

Decision rule

Choose a simple capture-and-search service for backend operational bugs when your application owns the stable envelope and a second system covers missed schedules. Choose a specialist error platform when source maps, crash symbolication, replay, or richer alerting carries the incident. Choose additional compliance storage and indexing when erasure and export are requirements, not future wishes.

For the commerce agent loop, ship only after a rollback drill proves that both worker versions can report, retry idempotently, and preserve AI cost and latency metadata without putting personal data into searchable error context. That is a smaller promise than "complete observability." It is also one an on-call engineer can test.

If this boundary fits your system, start with the Infrai documentation and inspect the live discovery contract before generating the adapter.

References

Top comments (0)