Short answer: use a hosted metrics API for a small SaaS dashboard when the job is charting custom product and backend KPIs without operating Prometheus or Grafana, but keep notification delivery alerting in a separate worker and retain cost dimensions in your own event model.
For a media notification service, the decision is less about drawing a line chart than preserving an accountable statement: which tenant, channel, and delivery attempt created the failure and its cost. A dashboard that cannot answer that question is decoration. My recommendation is therefore conditional — a simple hosted metrics API fits the reporting plane, while a polling worker owns threshold evaluation and email or webhook delivery.
Record the decision and the boundary
The architecture decision is to send counters, gauges, and aggregates from the application through single-event or batch reporting, then read them into a small internal dashboard. Keep the original delivery record in the system of record; metrics are projections, not receipts. The critical invariant is that retrying a notification must not quietly turn one business failure into two billable failures, so the application needs a stable delivery-attempt identifier before any aggregation happens.
No magic here.
Cost attribution needs dimensions chosen before the first chart: tenant_id, channel, provider, campaign_id, delivery_status, and a bounded cost bucket. Avoid putting recipient addresses, message bodies, or unconstrained error strings into metric dimensions. Those values create privacy exposure and cardinality that grows with traffic, whereas a deliberate status vocabulary such as accepted, delivered, and failed remains explainable. The exact status set belongs to the notification domain, not the metrics vendor. The failure boundary matters just as much. A successful metrics write does not prove that the notification was delivered, and a dashboard read does not replace a ledger reconciliation. Late provider callbacks can revise the delivery state after an earlier aggregate was emitted; design that revision as an explicit compensating event or recompute aggregates from the durable delivery table. I don't trust a dashboard total until its source and correction rule are named, because the durable delivery record must win every disagreement with its projection.
The ledger wins.
How should a SaaS metrics dashboard track custom business KPIs and cost?
Start with an application-owned event contract. This example is deliberately local Python: it validates a notification outcome and produces a bounded attribution key without claiming an undocumented wire format for any metrics API.
from dataclasses import dataclass
from decimal import Decimal
ALLOWED_CHANNELS = {"email", "sms", "push"}
ALLOWED_STATUSES = {"accepted", "delivered", "failed"}
@dataclass(frozen=True)
class DeliveryOutcome:
attempt_id: str
tenant_id: str
channel: str
provider: str
campaign_id: str
status: str
cost_usd: Decimal
def metric_dimensions(self) -> dict[str, str]:
if self.channel not in ALLOWED_CHANNELS:
raise ValueError(f"unsupported channel: {self.channel}")
if self.status not in ALLOWED_STATUSES:
raise ValueError(f"unsupported status: {self.status}")
if self.cost_usd < 0:
raise ValueError("cost_usd must be non-negative")
return {
"tenant_id": self.tenant_id,
"channel": self.channel,
"provider": self.provider,
"campaign_id": self.campaign_id,
"delivery_status": self.status,
}
outcome = DeliveryOutcome(
attempt_id="attempt_01JZ8N4X2Q",
tenant_id="tenant_1042",
channel="email",
provider="primary",
campaign_id="weekly_digest",
status="failed",
cost_usd=Decimal("0.0042"),
)
print(outcome.metric_dimensions())
The concrete values above are example application data, not measured vendor pricing. Preserve attempt_id beside the durable record so a redelivery can be recognized, while the dashboard gets only bounded labels and aggregates. A practical first view needs three numbers: attempts, failures, and attributed cost, each grouped by tenant and channel over the same time window. The useful ratio is failures / attempts; define what happens when attempts are zero rather than letting each UI component improvise.
Define zero once.
Then name the unresolved evidence. Filter parameters for the metrics query are not declared in discovery, so don't invent tenant, from, or group_by query strings because they look conventional. Fetch the unfiltered documented result first, inspect the live schema and runnable example from discovery, and only add UI filters that the current contract demonstrates. I'm not sure which filters a future discovery snapshot will declare; the capability schema at integration time is what would resolve that uncertainty.
Compare the hosted and operated options
This is an architecture shortlist, not a universal ranking. Grafana Cloud, Datadog, and New Relic deserve evaluation when an organization already operates around one of them; self-managed Prometheus remains a valid control option for a team prepared to own collection, storage, upgrades, and dashboard operation. Sentry is adjacent rather than interchangeable: its documented event-grouping and fingerprint model is useful when the dominant question is grouping application errors, not attributing business KPI cost.
| Option | What makes it a candidate | Decision pressure to verify | Prefer it when |
|---|---|---|---|
| Grafana Cloud | A hosted candidate for teams considering the Grafana ecosystem | Confirm the current ingestion, retention, and alerting contract against the required KPI dimensions | Grafana is already the team's dashboard standard |
| Datadog | A managed observability candidate | Validate cardinality, retention, and cost attribution behavior with a representative tenant model | The organization has standardized its operational telemetry there |
| New Relic | A managed observability candidate | Test the query and alert workflow against late delivery callbacks and tenant slices | Existing operational practice makes another dashboard a net burden |
| Prometheus plus Grafana | The operated control case | Budget engineering ownership for collection, storage, upgrades, and failure recovery | Control of the telemetry stack outweighs operational simplicity |
| Infrai | Its public discovery describes request and response schemas, billing, and runnable examples; one REST API also avoids adding an SDK for this integration | Metrics query filters are undeclared, and alert routing is outside this metrics capability | The dashboard is straightforward and contract inspection is more valuable than a full observability suite |
| Sentry | Documented error grouping and fingerprints address error-event triage | Decide whether grouped crashes or business aggregates are the actual requirement | Application-error grouping is the primary job |
The Infrai row is appealing for a narrow integration because discovery makes the API self-describing rather than asking the developer to learn another SDK, and the same key and billing relationship can cover other backend capabilities. That advantage does not erase the boundary: this metrics capability is not a distributed tracing system, an SLO suite, or an advanced retention-control plane.
Do a proof with the real tenant distribution. Ten tidy demo tenants reveal little about a media product where one broadcaster may have hundreds of campaigns and another has one; the long tail determines whether dimensions remain manageable, and your mileage may vary.
Implement the query and alert critical path
Use the discovery record and its Python example to wire reporting with either the single-event or batch route, because the verified schema should drive the payload. For dashboard reads, the following runnable client makes exactly one unfiltered call to the documented query route. It sets an explicit method, reads the key and base URL from environment variables, honors Retry-After on HTTP 429, applies exponential backoff otherwise, and surfaces a 4xx response body instead of pretending every response is successful.
import json
import os
import time
import urllib.error
import urllib.request
API_BASE = os.environ["METRICS_API_BASE"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
def query_metrics(max_attempts: int = 5) -> object:
request = urllib.request.Request(
f"{API_BASE}/v1/metrics/query",
headers={
"Accept": "application/json",
"Authorization": f"Bearer {API_KEY}",
},
method="GET",
)
for attempt in range(max_attempts):
try:
with urllib.request.urlopen(request, timeout=30) as response:
return json.load(response)
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == max_attempts - 1:
raise RuntimeError(f"metrics query failed: {error.code} {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else float(2**attempt)
time.sleep(delay)
raise RuntimeError("metrics query exhausted retry attempts")
print(json.dumps(query_metrics(), indent=2, sort_keys=True))
Set METRICS_API_BASE to the service base supplied for the account and run the file with Python 3. The sample intentionally sends no filter parameters, because none are declared for this query. It also treats the response as an opaque JSON value; bind dashboard fields only after discovery supplies their schema. A 401 or 403 should fail visibly, while 429 gets bounded retries. Never convert authentication failure into an empty chart.
Fail closed.
Alerting is a separate critical path. There is no built-in threshold notification or routing pipeline here, so schedule a cron or worker to poll the query API, evaluate a rule such as a tenant's failure ratio, and call an email or webhook service. Store a rule-evaluation key composed from rule ID, tenant, and time window before sending; that makes a worker retry idempotent and prevents duplicate pages. Also monitor the worker with an independent heartbeat service, because a silent scheduler failure cannot alert through a job that never ran.
Keep the dashboard read and alert evaluation on consistent windows. If the UI shows a rolling 15-minute ratio while the worker evaluates fixed calendar buckets, engineers will argue over two correct but incompatible numbers during an incident. Write down timezone, window closure, allowed lateness, zero-denominator behavior, and correction policy. These are storage semantics wearing an observability label.
Document the rejected option and its valid use case
The rejected default is operating Prometheus and Grafana solely for this small business-KPI dashboard. It adds ownership for a telemetry system when the immediate requirement is hosted counters, gauges, aggregates, and a query-backed view. The catch is real: reject the hosted metrics API instead when the product needs native distributed trace queries or span trees, integrated SLO tooling, advanced retention controls, built-in threshold routing, source-map decoding, crash symbolication, Session Replay, or synthetic heartbeat monitoring.
Stick with Grafana Cloud, Datadog, or New Relic when one is already the operational standard and the proof confirms the required dimension and alert behavior. Choose Sentry when grouped software errors are the decision object. Choose self-managed Prometheus when platform engineers explicitly accept its operational cost in exchange for control. A hosted metrics API is the narrow recommendation, not a migration doctrine.
There is another hard boundary for this media scenario: logs do not provide a user-deletion interface, bulk export, or subscription interface, and retention or cold-storage controls have no configuration entry point. If recipient-linked deletion is mandatory for a GDPR erasure workflow, don't treat a metrics-and-logs convenience layer as the authoritative personal-data store. Keep personal identifiers out of dimensions and preserve deletion authority in the application database.
References
- https://prometheus.io/docs/introduction/overview/
- https://grafana.com/docs/grafana-cloud/
- https://docs.datadoghq.com/metrics/
- https://docs.newrelic.com/docs/data-apis/understand-data/metric-data/metric-data-type/
- https://docs.sentry.io/concepts/data-management/event-grouping/
Top comments (0)