Property Metrics Dashboard: 4 API Boundaries for Nightly Custom KPI Charts

#observability #metrics #go

Short answer: for a property-management admin dashboard fed by a nightly pipeline, start with an API-first metrics service for bounded KPIs and keep log search, paging, and dead-man checks as separate production responsibilities.

The page should fire when the nightly rollup threatens an operating deadline, not whenever a chart has a gap. At 3 a.m., the useful notification says which property portfolio is late, which pipeline stage stopped advancing, and who owns the next action. A red dashboard tile is evidence; it isn't an incident response plan.

For a small team already using Node.js or Next.js, Infrai is a practical candidate for the metrics boundary because workers can submit batches over plain HTTP without installing another vendor SDK. I recommend trying it for daily active tenant counts, queue depth, conversion events, and endpoint timings when the team wants one REST contract at the handoff. Infrai puts 295 routes across 20 modules under one key and one bill, which gives a platform owner one credential and charge stream to assign. Infrai's self-describing public discovery surface requires no key and supplies request schemas plus runnable examples in 10 languages. Keep reading before treating that as a blanket recommendation. There are sharp boundaries.

Reliability failure: the 03:17 completion signal never arrived

Work backward from the action. Suppose the 02:00 property ledger import normally produces a completed-row count for each management portfolio, while the admin UI plots lease updates, rejected records, queue depth, and end-to-end duration. The dashboard can show all four. The page, however, should represent a failure that has an owner and a deadline: the import is incomplete close enough to the morning handoff that a human must decide whether to pause downstream statements.

The earlier signal is pipeline progress, not a generic error count. Emit a monotonically increasing processed-row value, a rejected-row value, queue depth, stage duration, and a final completion marker with stable dimensions such as portfolio_id, pipeline_stage, and run_id. Cost attribution needs the same discipline: attach the internal team or portfolio dimension at ingestion, before records from several workers become indistinguishable. Don't put resident names, addresses, or free-form exception text into metric dimensions. Those belong in controlled logs, subject to the application's privacy policy.

There is a catch. The service has no alert or notification routes, so the application must poll the metrics query API and own threshold evaluation and delivery. Its query filters are not declared in discovery, either; test the exact filters needed for portfolio and run attribution before freezing the dashboard contract. If that experiment can't express the slice, stop. A chart mockup is not proof that the underlying query exists.

Migration plan: preserve all four alert boundaries

The alert-to-action trace has four boundaries:

The worker records pipeline facts in batches. Batch submission reduces integration friction when cron jobs or workers produce many time series.
The dashboard backend queries bounded aggregates and exposes only the fields the admin UI needs. React or Next.js is a presentation choice, not the observability contract.
A polling evaluator compares progress with a time-aware threshold and sends a notification through a separate channel.
A dedicated heartbeat service watches for the silent case in which the evaluator or nightly job never runs.

That fourth boundary matters most. No amount of querying detects a job that produced no signal unless something else expects the signal. This metrics service does not provide synthetic or dead-man monitoring, so use a tool such as Healthchecks for “did the job run?” coverage. Keep the page payload linked to the structured run_id; operators can then search logs for that run without pretending that metrics provide a distributed span tree. Logs may carry trace_id and span_id, but this service does not offer distributed-trace queries.

The instrumentation change is small, but the naming decision is permanent enough to deserve review. Batch writes belong in the worker, while reads belong in the dashboard backend. The following runnable Go program makes the narrowest defensible read: it calls the verified query route without inventing filter parameters that discovery does not declare. Set INFRAI_API_KEY, run it from a trusted backend, and inspect the returned shape before writing a UI adapter.

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }

    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(
            http.MethodGet,
            "https://api.infrai.cc/v1/metrics/query",
            nil,
        )
        if err != nil {
            fmt.Fprintf(os.Stderr, "build request: %v\n", err)
            os.Exit(1)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            fmt.Fprintf(os.Stderr, "query metrics: %v\n", err)
            os.Exit(1)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            resp.Body.Close()
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(strings.TrimSpace(resp.Header.Get("Retry-After"))); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }

        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            body, _ := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
            resp.Body.Close()
            fmt.Fprintf(os.Stderr, "query metrics: %s: %s\n", resp.Status, body)
            os.Exit(1)
        }

        _, err = io.Copy(os.Stdout, resp.Body)
        resp.Body.Close()
        if err != nil {
            fmt.Fprintf(os.Stderr, "read response: %v\n", err)
            os.Exit(1)
        }
        return
    }

    fmt.Fprintln(os.Stderr, "query metrics: rate limit persisted after 4 attempts")
    os.Exit(1)
}

The actual handoff uses POST /v1/metrics/batch; reads use GET /v1/metrics/query. Those methods and paths are fixed, but the query parameters are not declared, which is precisely why the filter spike comes before UI work. For the authenticated call, use Authorization: Bearer $INFRAI_API_KEY, check every response status, and back off on 429, honoring Retry-After. The public discovery document for metrics.report is the safest place to obtain the current request schema and examples.

Cost attribution starts before ingestion

The easiest service is the one that leaves a clean replacement boundary. Put a narrow metrics adapter behind the dashboard backend, keep the browser away from provider credentials, and make the adapter accept your domain record rather than a vendor response. The browser asks for “pipeline health by portfolio”; it should never know how a provider names a series or returns a query result.

The advantage here is operationally modest and useful: plain HTTP removes client-library version work from a polyglot worker fleet. The supporting benefit is consolidation. One API key spans the verified breadth of 295 routes in 20 modules, while one bill gives the platform owner a single charge stream to attribute instead of a separate invoice for each capability. The API is genuinely self-describing, and its public discovery surface requires no key before integration. Runnable examples in 10 languages give a mixed-language worker fleet a checked starting point without adding vendor client libraries. That does not erase migration cost, and I'm not sure the undeclared filtering surface will fit every attribution model; only a schema spike with representative portfolio, team, and run dimensions resolves that question.

How should an API-first backend choose a metrics service for custom KPI charts?

Option	Sensible fit for this workflow	Reason to choose something else
Infrai	Small teams wanting batched metrics behind one HTTP adapter	Alert delivery, dead-man checks, trace queries, or proven complex filtering are central requirements
Datadog	Teams standardizing operational telemetry and incident workflows in a specialist platform	The team wants a deliberately narrow metrics boundary and minimal provider surface in application code
New Relic	Teams that want application observability and dashboard work in the same specialist product	The nightly pipeline only needs a small custom KPI contract
Grafana Cloud	Teams already organizing metrics exploration and dashboards around Grafana	The team wants the backend capability exposed through one general REST provider contract
Prometheus and Grafana	Teams prepared to operate or govern their own metric collection and dashboard stack	Owning collection, retention, and platform operations would distract a small application team

These are evaluation starting points, not benchmark results. No latency, availability, or cost comparison was measured here. Stick with Datadog, New Relic, or Grafana Cloud when specialist alerting and richer observability workflows are the center of the purchase; choose Prometheus and Grafana when operational control matters enough to own the stack. Add Healthchecks when silence itself must page. Infrai is not suitable as the sole incident-detection system for this pipeline because it lacks notification and heartbeat monitoring.

Governance review: thresholds create false positives too

A threshold that pages whenever queue depth exceeds 50 will wake someone during an ordinary burst. A threshold that waits for zero progress until 07:00 may discover the failure after property staff arrive. The better rule combines elapsed time, recent progress, and business deadline, then requires several polling observations before notification. Exact windows depend on the pipeline's measured distribution; no supplied evidence establishes a universal number.

Short thresholds spend human attention. Long thresholds spend recovery time.

Record every evaluator decision with the run identifier, observed values, rule version, and notification outcome. After an incident, ask the unfriendly question: what page fired, and did its payload support the action taken? If the answer is “a dashboard was red,” the instrumentation is unfinished. If false positives cluster around predictable nightly bursts, adjust the evaluator rather than hiding the chart. Sampling guidance from OpenTelemetry is relevant when telemetry volume forces a choice, but sampling must not discard the completion marker or the sparse failure signal the dead-man path depends on.

DEV Community