The operational constraint is simple: an API key must rotate without taking the service down, while an internal API usage dashboard still proves which credential was used. Keep raw events for that audit trail.
Short answer: retain raw timeseries for auditability, expose rolled-up totals for normal dashboard reads, and make every rollup reproducible from immutable events.
That choice is less about chart speed than about evidence. During a key rotation, a total that says 18,240 requests is not enough. Operators need the time window, key fingerprint, route, response class, and ingestion sequence behind the number.
Should a Node.js usage dashboard keep raw timeseries or rolled-up totals?
Use both, with different contracts. Raw events answer “what happened?” and rolled-up rows answer “how much happened in this bucket?” A dashboard endpoint should not scan an event table for every refresh, but the rollup must never become the only copy.
A useful event shape is deliberately boring:
type ApiUsageEvent = {
id: string;
occurredAt: string;
receivedAt: string;
keyFingerprint: string;
route: string;
statusClass: 2 | 3 | 4 | 5;
requestCount: number;
};
The two timestamps matter. occurredAt supports a timeseries; receivedAt exposes delayed ingestion after a deploy or network partition. The fingerprint is non-secret, so the dashboard can distinguish old and new credentials without putting the key itself into logs.
I initially considered deleting raw rows after a seven-day rollup. That makes retention look cheap, but it destroys the evidence needed to explain a late request during a rotation. Keep a bounded raw-retention policy instead, and document exactly what an auditor can reconstruct after it expires.
The rotation workflow is an audit workflow
Treat rotation as a state transition, not a configuration edit. Create the replacement credential, deploy it behind a feature flag, observe traffic tagged with both fingerprints, then revoke the old credential only after the overlap window closes. Each transition should emit an event with an actor, reason, and change ID.
The dashboard can then show a small timeline:
const rotationWindow = {
changeId: "chg_2026_0912_014",
oldKey: "sha256:old-fp",
newKey: "sha256:new-fp",
startsAt: "2026-09-12T08:00:00Z",
revokeAfter: "2026-09-12T08:30:00Z"
};
A late event is not automatically a breach. It may be a queued request whose occurredAt precedes revocation. That is why the UI needs both event time and ingest time, plus a visible “data through” watermark. Without that watermark, stale cached totals look authoritative.
Cache and schedule boundaries in a Node.js service
Cache only derived views. A short-lived cache for the last 24 hours keeps the dashboard responsive, while the raw store remains the source for reconciliation. Include the query window and rollup version in the cache key; otherwise a schema change can silently serve an old aggregation.
A scheduled worker should process a closed time window, such as the previous five-minute bucket. It records a high-water mark, writes an idempotent aggregate, and advances the mark only after the write commits. If the worker runs twice, the same event IDs must not double-count. If it runs late, the next run should catch up instead of skipping the bucket.
The failure case I design for is a deploy that pauses the worker while requests keep arriving. Suppose buckets 08:00, 08:05, 08:10, and 08:15 are open when the process stops, and the cache still contains the 08:00 total. On restart, the worker should close 08:00 first, persist its event-ID watermark, then close each later bucket in order; a retry after a database timeout must re-read the same IDs and produce the same sum. The API can return the cached 08:00 value with dataThrough: 08:00, but it should not pretend that 08:15 is current. That explicit lag is useful evidence during key rotation because an operator can tell the difference between “no traffic” and “not processed yet.”
| Layer | Answers | Failure to watch |
|---|---|---|
| Raw events | Which key, route, and status occurred? | Missing or duplicated IDs |
| Cache | What should the operator see quickly? | Stale data without a watermark |
| Rollup | How many requests were in each bucket? | Double counting or skipped windows |
PostgreSQL is a sensible baseline when transactional writes and SQL investigation matter. Redis is useful for expiring dashboard fragments, but it is a poor audit ledger by itself. ClickHouse can handle large analytical scans, yet it adds an operational boundary that a small team may not want. These are trade-offs, not rankings.
What should be measured before copying this architecture?
Measure freshness, reconciliation, and rotation visibility together. Track p95 dashboard latency for cached and uncached reads, the age of the newest occurredAt event, rollup lag, duplicate-event rejects, and the percentage of requests carrying a known key fingerprint. During a staged rotation, verify that the old-key count reaches zero after revocation and that the raw-to-rollup sum matches for every closed bucket.
One concrete test is intentionally uncomfortable: stop the scheduler for 20 minutes, continue serving traffic, then restart it. The correct result is a visible freshness gap followed by a deterministic catch-up, with no change to historical totals. If the chart quietly smooths over that gap, the dashboard is hiding an operational fact.
Your mileage may vary on retention length. Regulatory requirements, event volume, and the cost of replay decide it; there is no universal seven-day answer.
The catch is storage and query discipline. Raw events increase retention work, and a dual-path read model gives you more states to monitor. This design is not suitable when you only need a disposable product metric and have no audit obligation; a single pre-aggregated table may be enough there. Stick with a simpler total when the business cannot act on per-request evidence.
For an internal developer tool, key rotation is exactly when a dashboard earns its keep. Keep the evidence immutable, make summaries disposable, and let the UI state how fresh its numbers are.
References
- https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- https://nodejs.org/api/crypto.html
- https://www.postgresql.org/docs/current/transaction-iso.html
- https://redis.io/docs/latest/develop/data-types/strings/
- https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree
Top comments (0)