A self-hosted notification dashboard built with Metabase, Redash, or Supabase charts has one expensive requirement that changes the build: every failure must be attributable to a tenant, channel, and delivery attempt without turning raw event history into an unbounded analytics bill. My choice for a small Node.js SaaS is to own the metric contract and aggregation table, then treat the chart layer as replaceable. The dashboard should query bounded rollups, not delivery logs.
TL;DR: start with a self-hosted query-and-chart process when the existing database can serve pre-aggregated tenant metrics within an explicit latency budget. Consider a managed metrics API when operating ingestion, retention, access control, and query capacity consumes more founder hours than the product work it enables. The cheapest option is the one with the lower total burden per useful dashboard view, not the smallest invoice.
1. What exactly should a delivery KPI count?
A graph cannot repair an ambiguous event model. For a notification service, one logical notification may produce several delivery attempts. Counting attempts as notifications inflates volume; counting only final failures hides retries and their cost. I would define the grain before selecting any dashboard tool.
The smallest useful record has a stable notification ID, a unique attempt ID, a tenant ID, a channel, a status, a timestamp, and an integer cost allocation unit. The unit can represent an internal accounting weight rather than currency. That keeps changing provider prices out of the metric contract.
type DeliveryStatus = "accepted" | "delivered" | "failed";
type Channel = "email" | "sms" | "push";
type DeliveryAttempt = {
notificationId: string;
attemptId: string;
tenantId: string;
channel: Channel;
status: DeliveryStatus;
occurredAt: Date;
allocationUnits: number;
};
This distinction matters during an incident. A tenant with 100 failed attempts could have 100 failed notifications, or one notification retried 100 times. Those are different product and cost failures. Preserve both identifiers, then aggregate each measure deliberately.
2. Should Metabase, Redash, or Supabase charts serve in-app SaaS KPIs?
The comparison is not really "self-hosted charts versus an API." It is a question of where ingestion, aggregation, authorization, and retention live. Metabase and Redash are named in this space as self-hosted query-and-visualization options; Supabase Charts is another charting path in the reader's candidate set; a managed metrics API moves more of the metrics lifecycle outside the application. Those labels do not settle who owns the data contract.
Use one scorecard for every candidate:
| Boundary | Question to answer | Failure to avoid |
|---|---|---|
| Write path | Can notification delivery continue if analytics is unavailable? | Dashboard trouble blocking sends |
| Attribution | Is every rollup keyed by tenant and channel? | Shared costs with no owner |
| Query path | Are in-app users restricted to their own tenant? | Cross-tenant disclosure |
| Retention | Are raw attempts and rollups retained separately? | Paying to keep detail that no decision uses |
| Operations | Who handles upgrades, backups, capacity, and recovery? | Hidden founder-hour cost |
This makes the trade-off clear. A self-hosted process spends operating time to retain control of the query path. Its limitation is that upgrades, backups, and query capacity remain your work, so it isn't a fit when those chores regularly displace releases. A managed process exchanges some control for outsourced operations; it isn't a fit when its ingestion or retention boundary cannot preserve the tenant-level allocation model. Neither choice removes the need for tenant isolation or a stable semantic definition.
Ship weekly. If a dashboard choice needs a new event pipeline, a second authorization model, and custom embedding code before the first useful graph, it has missed that cadence. Start with the boundary that can be replaced.
3. Build the smallest bounded Node.js rollup
The write path should record delivery attempts independently of dashboard availability. A scheduled worker can then produce hourly rollups. Upsert by bucket, tenant, and channel so a retry of the aggregation job replaces the same logical result rather than adding another copy.
type HourlyDeliveryKpi = {
bucketStart: Date;
tenantId: string;
channel: Channel;
attempts: number;
delivered: number;
failed: number;
allocationUnits: number;
};
interface KpiStore {
replaceHourly(row: HourlyDeliveryKpi): Promise<void>;
readHourly(input: {
tenantId: string;
from: Date;
to: Date;
}): Promise<HourlyDeliveryKpi[]>;
}
async function refreshHour(
store: KpiStore,
attempts: DeliveryAttempt[],
bucketStart: Date
): Promise<void> {
const groups = new Map<string, DeliveryAttempt[]>();
for (const attempt of attempts) {
const key = `${attempt.tenantId}:${attempt.channel}`;
groups.set(key, [...(groups.get(key) ?? []), attempt]);
}
for (const group of groups.values()) {
const first = group[0];
await store.replaceHourly({
bucketStart,
tenantId: first.tenantId,
channel: first.channel,
attempts: group.length,
delivered: group.filter((x) => x.status === "delivered").length,
failed: group.filter((x) => x.status === "failed").length,
allocationUnits: group.reduce((sum, x) => sum + x.allocationUnits, 0)
});
}
}
The interface is intentionally dull. Good. It lets the application read the same rows through a self-hosted visualization process, its own in-app chart component, or an adapter backed by a managed metrics API. No chart package gets to redefine "failed."
For the in-app endpoint, derive tenantId from the authenticated server-side context. Do not accept it as a trusted browser parameter. Bound from and to, cap the bucket count, and return only aggregate rows. Those controls belong to the product even if a separate tool renders internal operational views.
4. Measure founder hours alongside infrastructure usage
A one-person SaaS cannot call operational labor free. Track the hours spent each month on upgrades, broken queries, access reviews, backups, schema changes, and dashboard support. Then track dashboard usefulness: views that lead to a retry policy change, a tenant conversation, or a channel investigation.
Use revenue per hour as the decision lens. A self-hosted stack can be sensible when it reuses a database and deployment process already under care. A managed metrics boundary can be sensible when it removes recurring undifferentiated work. The decision flips when the operational hours flip, even if raw compute use barely moves.
Do not use log volume as a proxy for KPI value. Logs preserve diagnostic detail, while this dashboard needs stable counts and allocation units. Datadog's public pricing page separates log ingestion from indexed-log retention, which is a useful reminder that collection and queryable retention can be distinct cost dimensions. It is evidence about a pricing model, not a reason to select a product.
Short version: price the whole loop.
5. Change the boundary only when scale supplies evidence
At larger volume, I would move aggregation from a periodic database scan to a durable consumer, keep raw attempts on a shorter retention policy than rollups, and precompute the tenant/channel windows the UI actually requests. I would also test late events and duplicate attempts explicitly. The metric contract stays put while the machinery changes underneath it.
There is a second scale axis: browser experience. Core Web Vitals defines user-facing measures including LCP, INP, and CLS, and recommends evaluating the 75th percentile. Those measures do not describe notification delivery health, but they can reveal whether the embedded dashboard itself is harming the application experience. Keep product-performance telemetry separate from delivery KPIs; compare them during a dashboard release.
The practical trigger to reconsider the boundary is observed work: aggregation misses its freshness target, tenant queries contend with transactional traffic, access controls become duplicated, or maintenance repeatedly crowds out the weekly release. Until one of those conditions appears, added architecture is speculation.
For this notification service, the durable asset is not the chart renderer. It is the small, tested contract connecting delivery attempts to tenant-level cost and failure metrics. Own that contract. Outsource the undifferentiated parts when the saved hours have somewhere better to go.
Sources
References:
Top comments (0)