A healthtech pricing dashboard has a harder constraint than ordinary product analytics: an operator must be able to reconstruct why a patient-facing estimate changed without treating a rollout counter as a financial record. The right design is a backend decision ledger for committed outcomes, joined to feature-flag exposure as release context.
TL;DR: flag statistics answer who was eligible for a variant and when; backend metrics answer what the pricing service actually decided. Keep both, correlate them with stable identifiers, and make the committed decision record the authority for incident reconstruction. A dashboard can aggregate that record, but it must never become the record itself.
Should a product analytics dashboard use backend custom metrics?
A flag evaluation is evidence about configuration. It can establish that rule version B was selected for a request under a particular rollout, but it does not by itself establish that the request passed validation, that the calculation completed, that a downstream adjustment was applied, or that the result was committed. Those distinctions matter when a pricing rule affects a care estimate and an incident reviewer needs a defensible sequence rather than a plausible correlation.
Consider two requests that evaluate the same variant. One is rejected because an input is incomplete. The other calculates an amount, retries after a timeout, and eventually commits once. A flag dashboard may count two evaluations. A product dashboard built only from that count cannot distinguish zero durable outcomes from one, nor can it explain which inputs and rule revision produced the committed result.
That is the gap.
This is the first boundary: exposure is context; a committed business decision is evidence. Neither replaces the other. Flag telemetry remains valuable for rollout shape, cohort imbalance, and timing, while backend events carry the facts required to replay the decision path.
Build an append-only decision trail
The backend should emit a decision event at the point where its durable state transition succeeds, using an idempotency key shared with the write operation. Emitting before commit risks describing a result that never existed. Emitting independently after commit risks losing the event between the database write and publication. A transactional outbox, or an equivalent atomic mechanism, closes that gap by committing the business row and an event-to-publish in the same transaction.
Exactly-once delivery is an unsafe assumption across a distributed pipeline. The useful engineering target is exactly-once effect: retries are expected, consumers deduplicate by event identity, and an audit query returns one logical decision. Short version: duplicates are normal.
Design for them.
A compact event can preserve reconstruction data without copying a patient record into the analytics plane. The following Go types separate correlation fields from controlled decision attributes; the values are illustrative schema fields, not an API contract.
type PricingDecision struct {
EventID string `json:"event_id"`
IdempotencyKey string `json:"idempotency_key"`
OccurredAt time.Time `json:"occurred_at"`
RequestID string `json:"request_id"`
SubjectRef string `json:"subject_ref"`
RuleRevision string `json:"rule_revision"`
FlagRevision string `json:"flag_revision"`
Variant string `json:"variant"`
InputDigest string `json:"input_digest"`
Outcome string `json:"outcome"`
AmountMinor int64 `json:"amount_minor"`
Currency string `json:"currency"`
}
type DecisionStore interface {
CommitOnce(ctx context.Context, event PricingDecision) (created bool, err error)
}
EventID identifies the emitted fact; IdempotencyKey identifies the logical operation. They should not be conflated, because an event may be republished while the business operation remains singular. RuleRevision identifies executable pricing logic, whereas FlagRevision and Variant identify rollout context. InputDigest supports equality checks during reconstruction without retaining raw clinical or identifying inputs in a broad analytics store. Its construction still requires a documented canonicalization scheme and protected keying material; a casual hash of low-entropy values is not a privacy control.
There is also a compliance boundary. GDPR Article 17 defines a right to erasure and enumerates exceptions, so retention cannot be reduced to “keep every event forever.” The event schema should minimize personal data, the subject reference should be separable from operational aggregates, and retention or erasure behavior should be reviewed against the actual legal basis and applicable exception. An immutable audit design does not cancel data-subject rights. It forces an explicit design for pseudonymization, access, retention, and deletion.
Join signals without changing their meaning
A useful dashboard joins three classes of signal, but does not collapse them into one metric.
| Signal | Question it answers | Authority | Common failure |
|---|---|---|---|
| Flag evaluation | Which rollout context was selected? | Flag evaluation log | Counting evaluation as a completed price |
| Decision event | What result did the backend commit, under which rule? | Decision ledger | Emitting before commit or accepting duplicates |
| Operational telemetry | Where did latency or errors occur? | Service telemetry | Using sampled traces as a financial total |
The join keys should support a narrow reconstruction path: request ID for one execution, idempotency key for one logical operation, event ID for publication deduplication, and revision identifiers for cohort analysis. A patient identifier is usually a poor universal join key because it broadens access and increases the consequences of retention. Use a scoped reference when the investigation genuinely requires subject-level lookup, and keep its resolution behind stronger authorization than aggregate dashboard access.
Counters still belong in the dashboard. For each rule revision and variant, show evaluated requests, validated requests, committed decisions, rejected decisions, retry attempts, duplicate publications suppressed, and decisions missing rollout context. The funnel exposes semantic gaps. If 10,000 evaluations yield 9,700 validations and 9,690 commits, the ten-event difference is an investigation target rather than a conversion-rate footnote; those numbers are an example of how the stages relate, not a reported benchmark.
The same discipline applies to amounts. Aggregate totals should derive from committed decision events and reconcile against the transactional system on a fixed schedule. Flag counts can segment the totals, but must not generate them. A late event should update the correct event-time window with a visible freshness marker, rather than silently moving into the current window and distorting the incident timeline.
Timing is evidence too.
Test the reconstruction path before rollout
A pricing-rule rollout should be tested as an evidence pipeline, not merely as a code path. Start with deterministic fixtures that pin the rule revision, normalized inputs, variant, expected outcome, and minor-unit amount. Then exercise the uncomfortable transitions: validation rejection after flag evaluation, retry after commit, duplicate outbox delivery, delayed publication, missing flag context, rollback to the prior rule, and erasure of the subject-linking record.
The acceptance test is concrete: given a request ID or authorized subject reference, can an investigator recover the selected variant, executable rule revision, input digest, committed outcome, and publication history without inferring any of them from neighboring traffic? Then run the reverse check: do aggregate counts reconcile to the source of record after duplicates and late arrivals are handled?
Two controls deserve separate alarms. A rise in pricing failures is a service incident. A rise in decisions without reconstructable evidence is an auditability incident, even if every request returned successfully. Combining those alarms hides the second condition until someone needs the history.
Keep them separate.
Sampling is another trap. Traces may be sampled for operational reasons, but the decision ledger used for reconciliation cannot inherit that sampling policy. Link a trace when available; do not require trace retention for the business fact to remain explainable. This is a deliberate storage trade-off: retain a small, controlled decision schema completely for its approved period, while keeping high-volume diagnostic detail under a different policy.
Roll out with an explicit promotion rule
Begin by producing decision events in shadow mode while the old pricing rule remains authoritative. Reconcile event counts and amounts against durable transactions, verify deduplication, and test authorized reconstruction. Next, expose the new rule to a bounded cohort and compare committed outcomes by rule revision, not merely evaluations by variant. Promotion should stop when reconciliation differs, evidence fields are missing, or the new revision breaches the predefined correctness and latency limits.
Rollback must preserve history. Do not rewrite prior events to make the dashboard look current; record the new configuration revision and the return to the earlier pricing rule, then let the timeline show both. This keeps the operational response compatible with later audit.
The final dashboard therefore has a modest role. It is a projection over evidence, optimized for questions and alerts. The decision ledger, transactional state, and rollout context retain distinct ownership, and their reconciliation rules are documented before traffic moves. That separation is what makes a new healthtech pricing rule explainable during the rollout and months afterward.
References
The retention and erasure discussion uses the legal text linked below; implementation-specific retention periods require legal and regulatory review for the relevant jurisdiction and data category.
Top comments (0)