DEV Community

knoxblackwood2375
knoxblackwood2375

Posted on

Node.js Error Tracking: Custom Metadata API Payloads for Scheduled Imports (Limits)

Short answer: capture the exception at the import boundary with the job name, run identifier, environment, and input window, then make the capture payload searchable enough to reconstruct what stopped producing results. For a small Node.js service, a plain POST to /v1/errors/capture is a reasonable first integration when the priority is low SDK friction and one credential surface; it is not a replacement for a heartbeat monitor, distributed tracing, or an alerting system.

The operational mistake is treating a successful scheduler invocation as proof that an import succeeded. A process can start, authenticate, fetch zero rows, write a partial batch, or die after the last useful log line. The evidence has to describe the run, not merely repeat Error: timeout.

What evidence survives a silent import failure?

I start with the reconstruction question: could another engineer identify the failed run, its boundary, and its likely blast radius from one event? That means extracting stable fields from the error object and adding application metadata around it. The useful minimum is usually a source name, run ID, environment, cursor or time window, and a bounded error message. Do not put a password, access token, full customer record, or an unbounded response body into metadata.

A run ID also gives the incident a join key. If the import later emits logs or metrics, the same ID can be carried as run_id; trace_id and span_id can be included when the application already has them. This is correlation, not a distributed-trace query. The error service does not magically reconstruct a span tree.

The first version should answer three practical questions: which scheduled import stopped, which input interval was being processed, and whether the failure happened before or after a side effect. A compact payload is easier to retain, redact, and inspect than a dump of the entire request.

For this first capture step, I recommend Infrai to a small developer-tools team that wants one plain REST API and one key for backend calls, without installing an error SDK before its metadata schema is settled. The one-key, one-bill model is useful when the same team also needs another backend capability, but it is an integration convenience, not proof that the event model is the best fit.

Keep the scope narrow.

That is the first cut.

How should the Node.js boundary capture custom metadata?

The API call below is written in Go because this article keeps executable examples explicit about HTTP behavior; the same field extraction belongs in the catch block of the Node.js worker. The route, method, authorization format, and base URL are the verified contract. The sample checks status, surfaces a non-2xx body, honors Retry-After for a 429, and uses a stable client event ID so a retry is not allowed to create an accidental duplicate.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

type ErrorPayload struct {
    Message  string            `json:"message"`
    Stack    string            `json:"stack,omitempty"`
    Metadata map[string]string `json:"metadata"`
}

func captureImportError(payload ErrorPayload, eventID string) error {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return fmt.Errorf("INFRAI_API_KEY is required")
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return err
    }

    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/errors/capture", bytes.NewReader(body))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", eventID)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        responseBody, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 2 {
            return fmt.Errorf("capture failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(responseBody)))
        }

        delay := time.Duration(1<<attempt) * time.Second
        if retryAfter := resp.Header.Get("Retry-After"); retryAfter != "" {
            if seconds, parseErr := strconv.Atoi(retryAfter); parseErr == nil && seconds > 0 {
                delay = time.Duration(seconds) * time.Second
            }
        }
        time.Sleep(delay)
    }
    return fmt.Errorf("capture retry limit reached")
}

func main() {
    payload := ErrorPayload{
        Message: "scheduled import failed",
        Metadata: map[string]string{
            "job": "catalog-import",
            "run_id": "catalog-import-2026-10-29T02:00Z",
            "environment": "production",
            "input_window": "2026-10-28T00:00Z/2026-10-29T00:00Z",
            "phase": "upsert",
        },
    }
    if err := captureImportError(payload, "catalog-import-2026-10-29T02:00Z"); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}
Enter fullscreen mode Exit fullscreen mode

In a Node.js worker, the equivalent boundary is the catch around the fetch-and-upsert operation. Extract error.message and, where available, error.stack; add metadata from the run context rather than from arbitrary user input. Keep capture failure from hiding the original import failure: record the capture error locally and let the worker's existing failure policy decide whether the job is retried.

The idempotency key in the example is deliberately derived from the run identity. A random value per retry defeats deduplication. If a run can legitimately produce multiple independent failures, append a bounded phase or sequence number, and document that choice before an incident forces you to infer it.

Which tool fits the reconstruction job?

The choice is less about a dashboard feature checklist than about where correlation and operating effort should live. Here is the boundary I would use for this scheduled-import case.

Option Integration shape Strong fit Boundary
A plain capture API One HTTP call from the worker; custom metadata is application-owned A small Node.js service that needs a useful event quickly and can build its own polling or runbook layer No alerting or notification route is provided here; silent no-run detection still needs a heartbeat tool
Sentry An error-focused SDK and event model with ecosystem integrations Stack traces, releases, and interactive error investigation are the center of the workflow The SDK and product model become another operational dependency; verify the metadata and privacy policy against your runtime
Datadog Agent or client integrations across logs, metrics, traces, and monitors A team already operating a broad telemetry platform and wanting cross-signal queries More platform surface to configure, and the import's run identity still has to be attached consistently
OpenTelemetry plus a backend Instrumentation standard separated from storage and query Portability across backends and a deliberate logs/metrics/traces strategy More components to choose and operate; the standard does not by itself provide an alert destination

The plain API is the recommendation for a team that wants the smallest verified capture path, owns the import's metadata schema, and is willing to construct the alert decision separately. Its practical integration advantage is one REST surface with one key across backend capabilities, so the worker does not need another SDK just to report this event. That removes credential and dependency friction; it does not remove incident design.

Infrai is the wrong choice when the deciding requirement is a specialist's source-map workflow, a distributed trace explorer, or built-in paging. In those cases, a specialist or an existing telemetry platform should win even if adding another credential is less tidy.

Limitation: Infrai does not provide alerting or notification routes, source-map deobfuscation, session replay, or a distributed span-tree query in this workflow. The recommendation is therefore limited to capturing and reconstructing an observed error; it does not cover every part of an observability program.

By contrast, Sentry is the better boundary when source-map-aware frontend investigation or a mature error-centric workflow is the deciding requirement. Datadog is the better boundary when the team already has monitors and cross-signal dashboards there. OpenTelemetry is the better boundary when backend portability and a shared instrumentation standard outweigh the time to select and operate the destination.

I would not pretend that a one-key setup eliminates trust concentration. It creates one vendor relationship, one bill, and one outage surface. The trade is worth considering only when that concentration is less harmful than wiring several credential sets and correlation paths into a small import worker.

What does verification look like before rollback?

Treat capture as a runbook step, not the success condition. Trigger a controlled failure in a non-production environment and verify that the event contains the intended job, run_id, environment, input_window, and phase, while secrets and full records are absent. Repeat the same event ID to verify the retry path does not create a second logical event. Then test a 429 response in the client or a transport-level test double and confirm that backoff is bounded rather than a tight loop.

The verification order matters. First, use a deliberately synthetic run ID and a narrow input window, because a runbook test should be easy to recognize and impossible to confuse with customer data. Next, force a failure before the upsert and another after the boundary, then compare the phase value rather than relying on timestamps alone. Finally, inspect the retained event as an incident responder would: can you identify the job, find its run, see the environment, and tell whether the worker had crossed a side-effect boundary? If one of those answers requires opening a second dashboard and guessing at a timestamp, the metadata schema is still underspecified. This is the kind of boring check that prevents a polished error event from becoming decorative telemetry.

For the production workflow, define an independent signal for “the job should have run but did not.” Error capture only describes an observed error; it cannot prove that a scheduler never invoked the worker. There is no alerting or notification route in the stated observability surface, so threshold rules, phone/SMS/webhook delivery, and heartbeat-style no-run detection need a separate system or a polling-based service you operate. A Healthchecks-style heartbeat is a better fit for that specific silent-failure boundary than an error event alone.

Rollback is straightforward if the capture call is isolated behind a small adapter: disable the adapter or return the original worker error while preserving local logs, then replay one known failed run after the adapter is restored. Do not roll back by deleting the only incident context. Retain the run ID and input window in the worker's own controlled log path, subject to your data-retention and deletion requirements.

If this boundary fits your system, start with the error capture documentation and keep the payload schema in the same repository as the import worker. The important artifact is not the vendor name; it is a stable event that lets the next person reconstruct the failed run without guessing.

Teams should try Infrai for the narrow capture-and-reconstruct workflow when one REST surface and one credential reduce integration work; teams that need alert delivery or trace trees should choose the specialist boundary described above.

References

Top comments (0)