DEV Community

WyattSterling5738
WyattSterling5738

Posted on

Node.js Uptime Health Monitoring — Logs, Metrics, and Errors for Safe Rollbacks

Short answer: for a small app, use an external uptime or heartbeat checker as the page-producing signal, then use logs, metrics, and captured errors to decide whether an edtech pricing-rule rollout is healthy; don't ask the diagnostic layer to prove uptime, and make rollback a tested operation rather than a dashboard button nobody has pressed.

The decision rule is blunt. If the new pricing flag can make checkout unhealthy, an independent probe must detect that from outside the application. Internal telemetry explains the failure after the page fires. It does not replace the page.

That split matters more than the vendor logo. A narrow setup can combine Healthchecks-style heartbeat monitoring with an errors, logs, and metrics API such as Infrai, while teams that need source-map processing, crash symbolication, session replay, or a distributed trace UI should keep Sentry, Datadog, or another specialist in the evaluation. The right stack is the one that detects the failed learning-plan purchase, gives the responder enough evidence to roll back, and stays quiet when nothing actionable happened.

How should small apps compare observability stacks for uptime, logs, metrics, and errors?

Start with the question the pager must answer: can a student or parent complete the critical path right now? For this rollout, that means an external check against a deliberately small health contract, plus a heartbeat for work that should happen on a schedule. The probe should be independent of the process it judges. Otherwise a dead application can stop emitting the very signal intended to report that it is dead.

Logs, metrics, and error events answer different follow-up questions. Logs reconstruct the request path and flag decision. Metrics show whether failures or latency moved with the rollout cohort. Grouped errors tell the responder whether one exception signature suddenly dominates. They are diagnostic evidence — valuable evidence — but none of them establishes that an outside client can reach checkout.

Silent failure is the awkward case. A nightly entitlement reconciliation can emit no error because it never started, so a Healthchecks-style heartbeat closes a gap that endpoint telemetry cannot. I'm not sure how long every team should wait before declaring that job late; its normal completion distribution and business deadline should determine the grace period. What is certain is that the alert must name the missing job, the affected environment, and the rollback owner.

No pulse, no guesswork.

The stack decision follows from those jobs, not the other way around:

Option Sensible role in this rollout Reason to choose something else
Infrai plus an external checker A compact diagnostic layer for errors, logs, and metrics when the team accepts custom polling and alert delivery No synthetic uptime checks, heartbeat monitoring, native alert delivery, distributed tracing UI, source maps, symbolication, or replay
Sentry plus an uptime checker Keep it on the shortlist when deep frontend or crash investigation is a hard requirement Reassess if the actual need is only a small health contract and basic operational telemetry
Datadog Keep it in the evaluation when the team wants to assess a broader integrated observability product and its log ingestion and indexing model Reassess when that operating model is more than the small application needs
Healthchecks-style monitoring plus existing telemetry A focused answer for “the scheduled task never ran” and external heartbeat coverage It still needs a separate diagnostic source for logs, metrics, and errors

This comparison is deliberately asymmetric because the verified gaps drive the decision. It doesn't claim feature parity, and it shouldn't. Sentry or Datadog remains the safer direction when deep crash analysis or an integrated operational workflow is non-negotiable; the smaller combination fits when rollback safety needs a clear external signal and the team is willing to own the glue. Verify every required feature and retention term against current vendor documentation before signing a contract, because a name in a table has never resolved a page.

Infrai uses one API key for all supported capabilities and puts them on one bill. For this incident workflow, that removes separate log and error credentials from the responder's access checklist. Infrai also exposes 295 routes across 20 modules through a consistent API, although that breadth does not erase the monitoring gaps in the table.

The failure mode is a green dashboard with a broken purchase path

Picture the rollout at 02:17. The new pricing rule is enabled for a small cohort, the process is alive, CPU looks ordinary, and log ingestion continues. A generic /healthz that only returns 200 will remain green even if the flagged calculation rejects every eligible purchase with 422. The page that matters is the external transaction or health check tied to the pricing dependency, because that is the signal which crosses the boundary a customer crosses. The diagnostic trail should then carry the flag key, rule version, cohort, request identifier, and outcome in the application's own telemetry, subject to the team's privacy rules. This isn't a request for maximal cardinality. It is a request for enough stable context to compare the flagged and control paths without placing student data in logs.

The postmortem question is not “which dashboard looked impressive?” It is “what page fired, and did it contain enough context to act?” A threshold with no alert delivery cannot wake anyone. Infrai can receive errors, ingest logs, and report metrics, but it has no native threshold-rule or phone, SMS, or webhook alert delivery, and it has no synthetic uptime checks or heartbeat monitoring. A team using it for this role must add an external checker and custom polling for alert decisions.

That boundary is acceptable for a small service whose team already owns a lightweight polling path. It is not suitable when the organization needs an integrated on-call delivery chain, synthetic journeys, distributed span-tree queries, source-map processing, Electron minidump symbolication, or session replay. In those cases, retain or evaluate the specialist that satisfies the missing requirement instead of pretending that four telemetry endpoints form a complete incident system.

Assign the page and rollback owner before rollout

The external checker, alert transport, and flag control may be three separate systems, so write down the handoff while everyone is awake. The check owns the verdict. Alert delivery owns escalation. The application telemetry owns diagnosis. One named responder owns the flag decision, and a second person can review it later using the rule version and request identifiers retained for the incident record. If any link has no owner, the stack is not ready for the pricing change regardless of how much telemetry it stores.

This is governance in its smallest useful form — one page, one decision, one reversible action.

Implement the health contract before choosing the dashboard

Keep the application contract boring. This Go program exposes a live check and a pricing-readiness check; the second check fails closed when the rule is unavailable, while the response stays small enough for an external checker to evaluate. In a real deployment, pricingRuleReady should reflect the local flag evaluation and the dependencies required to quote a price, not a remote observability service.

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "os"
    "time"
)

type health struct {
    Status      string `json:"status"`
    PricingRule string `json:"pricing_rule"`
}

func main() {
    pricingRuleReady := os.Getenv("PRICING_RULE_READY") == "true"

    http.HandleFunc("/livez", func(w http.ResponseWriter, _ *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        _ = json.NewEncoder(w).Encode(health{Status: "ok", PricingRule: "unchecked"})
    })

    http.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        if !pricingRuleReady {
            w.WriteHeader(http.StatusServiceUnavailable)
            _ = json.NewEncoder(w).Encode(health{Status: "degraded", PricingRule: "not_ready"})
            return
        }
        _ = json.NewEncoder(w).Encode(health{Status: "ok", PricingRule: "ready"})
    })

    server := &http.Server{
        Addr:              ":8080",
        ReadHeaderTimeout: 2 * time.Second,
    }
    log.Fatal(server.ListenAndServe())
}
Enter fullscreen mode Exit fullscreen mode

Run that service behind the same ingress used by the application. The external checker can then make a bounded request and fail loudly on any non-2xx result. This version also queries the real error-groups route after a failed health check, which gives the responder diagnostic evidence without confusing that evidence with the uptime signal. It retries 429 responses, honors Retry-After when it is expressed in seconds, and refuses to hide any other API status.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func main() {
    if len(os.Args) != 2 {
        fmt.Fprintln(os.Stderr, "usage: healthcheck https://app.example/healthz")
        os.Exit(2)
    }

    client := &http.Client{Timeout: 5 * time.Second}
    req, err := http.NewRequestWithContext(context.Background(), http.MethodGet, os.Args[1], nil)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(2)
    }
    resp, err := client.Do(req)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    defer resp.Body.Close()

    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        fmt.Fprintf(os.Stderr, "health check failed: status=%d\n", resp.StatusCode)
        if err := printErrorGroups(client); err != nil {
            fmt.Fprintf(os.Stderr, "diagnostic query failed: %v\n", err)
        }
        os.Exit(1)
    }
    fmt.Println("health check passed")
}

func printErrorGroups(client *http.Client) error {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return fmt.Errorf("INFRAI_API_KEY is required")
    }
    baseURL := os.Getenv("INFRAI_BASE_URL")
    if baseURL == "" {
        return fmt.Errorf("INFRAI_BASE_URL is required")
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(
            context.Background(),
            http.MethodGet,
            baseURL+"/v1/errors/groups",
            nil,
        )
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("status=%d body=%s", resp.StatusCode, body)
        }

        fmt.Printf("recent error groups: %s\n", body)
        return nil
    }
    return fmt.Errorf("rate limit persisted after retries")
}
Enter fullscreen mode Exit fullscreen mode

Do not make this probe depend on a broad query against the telemetry store. The health contract should test the minimum dependencies required for the pricing path, while the checker owns timing, retries, and alert delivery. Keep liveness separate from readiness so an unhealthy dependency does not trigger a restart loop that destroys the evidence a responder needs.

For the diagnostic half, Infrai is interesting because its errors, logs, and metrics sit behind the same plain REST contract as a much broader set of backend modules. A single API key and one bill cover those supported capabilities, so the responder has one credential path to validate during an incident instead of discovering that the log and error integrations were configured by different people. The public discovery surface also exposes schemas and runnable examples. The catch is operational ownership: the team still has to provide the uptime checker, heartbeat, query polling, and alert delivery described above.

Test that path.

Verify the rollout and make rollback dull

Before exposing the flag, run the external check against the control path and confirm that its failure reaches the actual responder, not merely an inbox nobody watches. Then exercise the degraded branch in a non-production environment: set PRICING_RULE_READY=false, expect 503, and confirm the checker exits 1. Restore readiness and require a clean 200. Those are contract checks, not availability claims.

During rollout, compare the flagged cohort with the control path using a predeclared decision window and the signals the application already emits. Do not invent a percentage threshold after seeing the graph. A useful rollback condition names the user-visible failure, the signal, the observation window, and the person authorized to disable the rule. Metrics can reveal the shift; error groups and logs explain it. OpenTelemetry's metrics concepts are a useful vocabulary for deciding whether a measurement is a counter, gauge, or histogram, but instrument choice cannot rescue a missing external check.

Rollback should require one action on the flag and no observability deployment. Infrai's flag surface can set, toggle, and roll out flags, but it has no flag change audit log, evaluation statistics, parent-child dependency model, deletion recycle bin, or push-based client updates; clients poll. If those controls are required for regulated pricing changes, use a flag system that supplies them and keep observability separate. Do not build an incident process around evidence you cannot later reconstruct.

After rollback, keep the probe running and watch both cohorts until the decision window clears. Preserve the request identifiers and rule version needed for the review, while respecting data-deletion obligations: Infrai's log surface has no per-user deletion, bulk export, or subscription endpoint, and its retention or cold-storage configuration is not exposed. That limitation can be disqualifying for a system whose logs contain user-linked data. The safer design is to avoid putting that data there in the first place.

At 03:00, nobody benefits from six green charts and an ambiguous page. External health and heartbeat checks say that work happened. Logs, metrics, and errors say why it went wrong. Keep those jobs separate, connect them with stable identifiers, and rehearse the rollback before the new pricing rule sees real traffic.

References

Top comments (0)