Short answer: use metrics APIs for cron-job results, API failures, and business-event charts, then add a separate Healthchecks-style heartbeat for scheduled imports that never start.
The deciding constraint is incident reconstruction. A customer-support import can fail loudly after processing 43 records, finish slowly with 2,100 records, or leave no result because the scheduler never invoked it. The first two cases produce measurements; the third produces silence. One dashboard can display both signals, but one signal system shouldn't be expected to create both.
That split is the experiment result I would carry from a FastAPI notebook into production: keep numeric evidence in metrics, enrich failure investigation with error events, and let an external heartbeat clock judge whether a run arrived on time. It is useful coverage for a small SaaS operations dashboard, not a claim of complete monitoring.
What must a cron-job incident timeline explain?
Start with the questions an on-call engineer will ask, not with charts. Did support_import begin in its expected window? Did it finish? How many tickets did it ingest? How long did it take? Was there a backlog, and did an API failure coincide with the drop in business events? Metrics fit success counts, failure counts, durations, backlog sizes, and error-rate trends. Error APIs can add failure counts and event detail while the time-series layer remains focused on charts.
Silence is different.
No ping, no proof.
A naive design increments import_success or import_failure at the end of each run and alerts when failures rise. I wouldn't ship that alone. If the scheduler, container, or invocation path never starts the code, neither counter changes; a flat chart can look calm during the exact incident the support team cares about. A heartbeat service reverses the test: it expects a ping by a deadline, so absence becomes evidence. This is the complement, not duplicate instrumentation.
For reconstruction, attach stable dimensions such as the job name and environment to measurements, and record business outcomes separately from transport outcomes. “The request returned” and “tickets became searchable” answer different questions. I'm not sure what lateness window fits every import because that depends on its real duration distribution; a useful eval is to replay recent schedules and choose a grace period that catches missed runs without paging on ordinary variance. Your mileage may vary.
How should a FastAPI backend metrics dashboard combine cron jobs, API failures, business events, and healthchecks?
Use four evidence lanes and join them by time, job identity, and, where available, a run identifier. The dashboard should show run counts and duration, API error trends, business outcomes such as imported-ticket counts, and heartbeat state. During an incident, the operator reads left to right: expected run, observed execution, external dependency behavior, customer-visible result.
| Evidence lane | Best signal | What it answers | What it cannot prove alone |
|---|---|---|---|
| Schedule completeness | Healthchecks-style heartbeat | Did the expected import report in? | How many records were processed |
| Job execution | Success, failure, and duration metrics | Did code run, and how did it finish? | Why a specific exception happened |
| API health | Error counts plus error events | Did dependency failures line up with the run? | Whether the scheduler invoked the job |
| Business result | Imported, skipped, and backlog metrics | Did useful customer-support data arrive? | Full request-level causality |
Keep the evaluation concrete. Given a missed run, an API rejection, a slow successful import, and a zero-result successful import, can an engineer classify all four from the dashboard without opening application logs first? That tiny incident-reconstruction harness is more informative than choosing a graph library because its screenshots look polished.
A focused Python implementation
The example below reads the verified metrics query route without inventing query filters, writes a local snapshot for the dashboard adapter, and sends the heartbeat only after the scheduled import has produced its result. It explicitly handles 429 using Retry-After when present and raises on other HTTP errors. Both URLs and the key come from environment variables.
import json
import os
import time
from email.utils import parsedate_to_datetime
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def retry_delay(response_headers, attempt):
retry_after = response_headers.get("Retry-After")
if retry_after:
try:
return max(0.0, float(retry_after))
except ValueError:
return max(0.0, parsedate_to_datetime(retry_after).timestamp() - time.time())
return min(2 ** attempt, 30)
def request_json(url, method, headers=None, attempts=5):
request_headers = {"Accept": "application/json", **(headers or {})}
for attempt in range(attempts):
request = Request(url, method=method, headers=request_headers)
try:
with urlopen(request, timeout=20) as response:
body = response.read()
return json.loads(body) if body else None
except HTTPError as error:
if error.code == 429 and attempt + 1 < attempts:
time.sleep(retry_delay(error.headers, attempt))
continue
reason = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"{method} {url} failed with {error.code}: {reason}") from error
raise RuntimeError(f"{method} {url} exhausted retries")
def refresh_dashboard_snapshot():
api_key = os.environ["INFRAI_API_KEY"]
base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
metrics = request_json(
f"{base_url}/v1/metrics/query",
method="GET",
headers={"Authorization": f"Bearer {api_key}"},
)
Path("metrics-snapshot.json").write_text(
json.dumps(metrics, indent=2), encoding="utf-8"
)
def report_completed_import():
request_json(os.environ["HEARTBEAT_URL"], method="POST")
if __name__ == "__main__":
refresh_dashboard_snapshot()
report_completed_import()
The order matters: call report_completed_import() from the actual successful completion path, after the import's business result is known. Don't ping merely because the process started. In a FastAPI deployment, a separate scheduler or worker can run this module while the web app reads metrics-snapshot.json through its normal dashboard adapter. No Infrai SDK is required; its plain REST surface works through Python's standard HTTP client, and the same key can cover a broad set of backend capabilities behind consistent conventions.
This sample intentionally leaves alert delivery outside the metrics client. Infrai has no alert or notification route for threshold rules, phone, SMS, or webhook delivery, so a team choosing it must poll query results and implement that alert path, while the heartbeat product owns missed-run notification. It also doesn't provide distributed trace queries or a span tree. Logs may carry trace_id and span_id for correlation, but that is not a tracing backend.
Which observability stack fits this boundary?
The right choice depends on how much monitoring machinery already exists. This isn't a winner-takes-all comparison; each option changes what the team operates and what it can reconstruct.
| Option | Strong fit for this job | Trade-off |
|---|---|---|
| Prometheus with Grafana | Teams already operating metric collection, queries, dashboards, and alert rules | More components and operational ownership for a small application team |
| Datadog | Teams wanting a managed, integrated observability suite | A broader platform may be more than a narrow import dashboard needs |
| Sentry | Application error investigation and grouping | Pair it with metrics and heartbeat coverage for the scheduled-run question |
| Healthchecks | Detecting that a cron-style job missed its expected ping | It does not replace duration, backlog, or business-event charts |
| Infrai plus Healthchecks | A small team wanting plain HTTP metrics and error APIs, with a separate missed-run clock | Requires polling and a self-built notification path; it is not suitable when native alerting or distributed tracing is mandatory |
Infrai uses one REST API and one key for 295 routes across 20 modules. That integration shape, rather than price, is the reason it belongs in this comparison: the import worker needs no monitoring SDK and can report metrics and capture errors without adding separate credentials and dependency upgrade cycles for each backend capability. Its self-describing public discovery surface requires no key and returns the request JSON Schema, response schema, billing metadata, and runnable examples for each capability; a build-time contract check can therefore catch an instrumentation mismatch before the scheduled worker is deployed. Those properties reduce configuration and schema drift in this specific workflow; they do not make the surface a full observability suite. The catch is real. Stick with Prometheus and Grafana when your team already owns that stack and wants its alerting model; choose Datadog when managed breadth is worth adopting a full suite; use Sentry as the center of gravity when exception investigation dominates. Healthchecks remains the purpose-built complement for silent scheduled-job misses in any of those combinations.
There are other boundaries. This approach has no synthetic probing or heartbeat monitoring inside the metrics API itself, and it does not offer source-map decoding, Electron minidump symbolication, or Session Replay. Those gaps may be irrelevant to a server-side support import, but they rule out treating the setup as universal observability.
What to measure before copying this design
Run the decision through an eval set built from incidents you actually need to distinguish. Measure whether the dashboard can identify a missed invocation, a started-but-failed run, an unusually slow completion, rising API failures, and a successful run with an unexpected business-event count. Also measure alert noise as the heartbeat grace window changes. Prompt and token costs don't drive this particular plumbing choice, though the same discipline applies: capture the smallest signals that answer the operational question instead of collecting data without an evaluation target.
Then test the human path — can the responder move from the missed heartbeat to the relevant time window and explain the customer impact? If the answer requires a trace waterfall, native threshold rules, or replay, choose a stack that supplies those features. If the answer rests on a compact timeline of expected run, execution metrics, error events, and business outcomes, the split design is a sensible notebook-to-production step.
Measure first.
Top comments (0)