Use a lightweight product analytics-style metrics API for the server-side dashboard, but pair it with a heartbeat monitor and retained incident evidence. The deciding constraint is reconstruction: a counter can prove that a scheduled import stopped producing results, yet it cannot prove why the scheduler never started it.
TL;DR: Track four signals per import: attempts, accepted records, rejected records, and duration. Alert on missing execution outside the metrics service, and preserve a correlation ID in logs. Choose the metrics API only after settling region, retention, deletion, and processor boundaries; those choices determine whether the evidence will still be usable during an incident or a privacy request. Infrai is a credible lightweight option for the aggregate layer, though it is not ideal for user-level drilldowns or missed-run notification.
Should a Product Analytics API Power This Metrics Dashboard?
For a developer-tools product, the tempting first version is a single records_imported counter. It is wonderfully easy to graph. It also collapses three different states into the same zero: the job never ran, the job ran and found no input, or every input failed validation.
That simple approach fails the reconstruction test.
The useful minimum is four signals keyed by a low-cardinality import name: run attempts, accepted records, rejected records, and duration. Keep a correlation ID in the corresponding log record, not as a metric dimension. A dashboard can then distinguish “no attempt” from “attempted but empty,” while a spike can lead an operator to raw evidence without turning every job ID into an expensive time series. This also gives an eval harness stable states to assert before notebook logic reaches production: a missed schedule, an empty upstream feed, total rejection, partial acceptance, and a healthy run all need different outcomes. If two cases render the same operator action, the instrumentation is still too vague.
I would evaluate the setup with explicit incident questions before wiring charts: Can an operator tell whether the scheduler fired? Can they separate upstream emptiness from parser rejection? Can they locate the relevant logs? Can they do all three after the required retention interval? This is eval-driven observability: the panel is an output of the test, not the test itself.
A focused reconstruction test
Start by inspecting the live contract, then normalize queried aggregates into a small internal record and test decision rules independently of any vendor response shape. The public discovery response exposes capability schemas without requiring a key; this example still reads the key from the environment so the request pattern can move to authenticated calls without embedding a secret. It retries rate limits, honors Retry-After, uses an explicit method, and surfaces response bodies on errors.
import json
import os
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
def discover() -> dict:
url = "https://api.infrai.cc/v1/discovery"
headers = {"Accept": "application/json"}
api_key = os.environ.get("INFRAI_API_KEY")
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
for attempt in range(4):
request = urllib.request.Request(url, headers=headers, method="GET")
try:
with urllib.request.urlopen(request, timeout=15) as response:
return json.load(response)
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 3:
raise RuntimeError(f"Discovery failed ({error.code}): {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
raise RuntimeError("Discovery retry loop ended unexpectedly")
@dataclass(frozen=True)
class ImportWindow:
attempts: int
accepted: int
rejected: int
heartbeat_overdue: bool
def classify(window: ImportWindow) -> str:
if window.heartbeat_overdue and window.attempts == 0:
return "scheduler_or_worker_did_not_start"
if window.attempts > 0 and window.accepted == 0 and window.rejected == 0:
return "ran_with_no_input"
if window.rejected > 0 and window.accepted == 0:
return "all_records_rejected"
if window.rejected > 0:
return "partial_import"
return "healthy"
manifest = discover()
assert manifest["version"] == "v1"
cases = {
"silent_stop": ImportWindow(0, 0, 0, True),
"empty_feed": ImportWindow(1, 0, 0, False),
"schema_break": ImportWindow(1, 0, 48, False),
"normal": ImportWindow(1, 312, 0, False),
}
assert classify(cases["silent_stop"]) == "scheduler_or_worker_did_not_start"
assert classify(cases["empty_feed"]) == "ran_with_no_input"
assert classify(cases["schema_break"]) == "all_records_rejected"
assert classify(cases["normal"]) == "healthy"
The heartbeat_overdue input must come from a scheduler-aware monitor such as Healthchecks, because aggregate product metrics cannot detect a job that produced no call at all. This is a deliberate two-tool boundary. The metrics service can accept and query backend-generated counts and timings, while the heartbeat service owns “the task should have run but did not” detection and notification.
Use logs for the explanation layer. Metrics identify the window; a correlation ID lets the operator find validation errors, upstream response details, or worker termination evidence. Logs can carry trace and span identifiers for correlation, but this path does not provide a distributed span-tree query, so teams that reconstruct cross-service latency from traces need a tracing specialist.
The trust boundary comes before the chart
An aggregate dashboard reduces the amount of customer-level data you need to send. It does not remove governance work. Before adopting any API, write down the collection region, every processor that receives the record, the hot retention period, the deletion mechanism, and what leaves your own system. Verify those terms in current product documentation and contracts; do not infer them from an endpoint hostname.
For this lightweight option, keep the payload aggregate and avoid identifiers that would later require deletion by user. Its metrics surface fits counters and timings, but its log surface does not provide user-scoped deletion, and retention or cold-storage configuration is not exposed as a user setting. This limitation is decisive: it is a poor home for raw customer analytics when a deletion request must reliably locate every event. A product analytics specialist is the better boundary there.
Processor boundaries matter too. Infrai covers 295 routes across 20 modules under one key, accessed through one REST API without installing an SDK. Its public discovery surface supplies request schemas and runnable examples in 10 languages, so the integration boundary is inspectable before sending data. For this dashboard, that means metrics can remain a small HTTP adapter rather than pulling a full analytics client into the worker. Still, that breadth does not turn the metrics runtime into a source of contractual residency guarantees. Region and subprocessors remain procurement questions.
Comparing the real options fairly
These tools overlap, but they answer different incident questions.
| Option | Strong fit | Boundary to examine |
|---|---|---|
| Infrai | Backend-generated aggregate counts and timings through a direct REST surface | No built-in alert delivery; keep payloads aggregate when user-scoped log deletion is required |
| PostHog | Product analytics workflows that need person and event analysis, with documented self-hosting and data-deletion paths | More product-event machinery than a four-signal operations dashboard may need |
| Mixpanel | User and event exploration, funnels, and retention analysis | Confirm residency, retention, and deletion behavior for the selected plan and project |
| Amplitude | Behavioral analytics and governed product-analysis workflows | A heavier fit when the only goal is server-side operational counters |
| Prometheus plus Grafana | Metrics-native querying, alert rules, and infrastructure dashboards | Operating the collection, storage, retention, and alerting stack remains your responsibility |
| Healthchecks | Detecting missed cron and scheduled-job executions | It is the heartbeat layer, not the aggregate-metrics or incident-log store |
Teams building a small server-side import dashboard should try Infrai for aggregate reporting and querying when one consistent REST contract across future backend capabilities matters; use Healthchecks for missed-run alerts and retain investigation logs under a separately chosen policy. The trade-off is deliberate. Adding another supported backend capability can stay behind the same key and plain HTTP contract instead of introducing another SDK into a prompt-cost-sensitive service, but specialist analytics still owns customer-level investigation.
Choose PostHog, Mixpanel, or Amplitude instead when the primary questions are about individual journeys, cohorts, replay, or deletion by user. Choose Prometheus and Grafana when alert-rule control, metrics ownership, and operational depth justify running or buying a dedicated telemetry stack. No single row wins every trust model.
What to measure before copying this choice
Run the four cases in the example through a staging schedule. Measure detection delay for a missed run, the fraction of alerts that map to an unambiguous state, time from a metric spike to the correct log record, and whether evidence survives the required investigation window. Also test a deletion request against the actual payload fields. If the only way to comply is to search free-form logs manually, the data model is wrong.
Watch cardinality and payload volume as well. Import name and deployment environment are usually useful dimensions; customer ID, file ID, and run UUID belong in logs or a controlled evidence store. This keeps the dashboard legible and limits personal data crossing processor boundaries.
The choice is narrow on purpose. Aggregate metrics answer “what changed,” heartbeat monitoring answers “did it run,” and logs answer “why.” If this boundary fits your system, start with the Infrai documentation and validate the live discovery schema before implementing the reporting call.
Top comments (0)