Short answer: attach the release, deployment environment, request trace context, error class, and a coarse experiment cohort to each error event; keep direct user IDs, raw request data, and free-form PII out of the telemetry path. Use a separate, access-controlled system for the rare investigation that requires identity, and expire the join key quickly.
That decision preserves the comparison a developer-tools team actually needs: did an experiment change the error rate or error mix for one tenant cohort relative to another? It also constrains cardinality before storage, makes deletion and retention rules intelligible, and avoids turning an error index into an accidental identity directory. The result is less investigative convenience per event, on purpose.
What metadata should error events carry for PII-safe EU and US tracking?
The event schema should answer four questions without identifying a person: what failed, which code was running, where it ran, and which experiment group received that code path. A compact envelope can contain error.type, service.name, service.version or a release identifier, deployment.environment.name, a timestamp, severity, trace ID, span ID, and a bounded cohort such as control or variant_b.
Treat the request ID as correlation data, not descriptive storage. W3C Trace Context defines interoperable trace and parent identifiers, while OpenTelemetry defines semantic conventions for naming telemetry attributes. Those standards make a trace ID preferable to copying a URL, query string, headers, or body into every error event. The trace still needs a retention limit and access policy; an opaque identifier isn't automatically harmless merely because a human can't read it.
User identity is the hard boundary. Don't attach an email address, account name, IP address, session token, authorization header, or raw user ID. If cohort analysis truly requires a stable unit, issue a purpose-specific, rotating assignment token in the experiment service and send only the cohort label to error telemetry. A tenant tier such as team can be useful, but a globally unique tenant ID creates nearly one distinct label value per customer and may permit re-identification when joined with another system. There is some uncertainty here: a field's privacy status depends on the surrounding data and the organization's ability to link it back to a person. A token described as “anonymous” can still be linkable. The appropriate privacy or legal review therefore examines the whole join path, not the spelling of one attribute. GDPR Article 5 supplies the useful engineering constraints: purpose limitation, data minimization, accuracy, storage limitation, and security. US obligations vary by jurisdiction and context, so a single region=us label does not settle the policy. For a concrete review, start with a hypothetical workspace that has ten seats but only one experiment assignment. The error event needs the assignment variant_b; it doesn't need the ten member IDs, their email domains, or a workspace slug. If support later investigates one member's report, the trace ID can be presented to an authorized lookup service during its short correlation window. After that window closes, the experiment counts remain useful because their unit was always the cohort, while the identity join is deliberately unavailable. That loss of convenience is the control working as designed.
The event should be boring. That's good.
No silent widening.
Invariants and failure boundaries
This architecture has three invariants. First, the ingestion allowlist is authoritative: unknown keys are rejected or dropped before durable storage. Second, every retained dimension has a bounded purpose and owner. Third, cohort comparison works from aggregates even after the short-lived correlation data expires.
The most common failure boundary is free-form text. Exception messages can contain a filename, an email address, a SQL fragment, or a user-supplied value even when the application never intended to record PII. Store a normalized error class and a reviewed message template; route an unredacted diagnostic payload, when one is operationally necessary, through a separately governed channel with narrower access and shorter retention. Redaction after indexing is too late because the sensitive value has already reached storage, replicas, and possibly alert notifications.
Cardinality is the second boundary. release=2026.08.16.3 has a manageable number of values if releases are finite. request.id is unique by design and should remain correlation context rather than an indexed grouping label. user.id and tenant.id are both expensive dimensions and weak cohort variables: they increase index fan-out while answering a question that experiment.cohort=control|variant_b answers directly.
Count before shipping. Suppose, only as an explicit planning example, that a service emits 2,000,000 error events per day. Adding a 36-byte identifier plus an estimated 14 bytes of key and encoding overhead adds 100,000,000 bytes per day before index, replication, and compression effects: 2,000,000 x 50. Over a 30-day retention window, that is 3,000,000,000 raw bytes for a field that the cohort query doesn't use. The exact stored amount will vary by backend and compression; the multiplication still exposes which assumption deserves measurement.
Sampling introduces a different failure. Uniform random sampling can estimate a common error rate, but it may erase a rare cohort or a low-volume release. Keep exact counters for the cohort-by-release decision cells, sample verbose exemplars separately, and record the sampling probability so estimates can be weighted. Never compare a fully retained control group with a sampled variant as if their event counts had equal exposure.
Decision table: signal quality versus noise
The schema choice is an architecture decision, not a request to collect every field and decide later. The comparison below assumes the goal is an experiment decision across tenant cohorts, not individual support case reconstruction.
| Option | Cohort signal | Cardinality and retention | Privacy boundary | Decision |
|---|---|---|---|---|
| Direct user and tenant identifiers on each event | Allows individual joins, but adds little to a cohort-rate comparison | Near-user or near-tenant cardinality; repeated bytes live for the event retention period | Broadest exposure and deletion surface | Reject for the experiment dataset |
| One-way stable hashes | Preserves joins and therefore preserves much of the linkability | Cardinality remains near-user; hashing doesn't reduce distinct values | Pseudonymization can reduce direct readability, but the join risk remains | Use only in a separately governed investigation dataset |
| Bounded cohort plus release, environment, and trace context | Directly supports rate and error-mix comparisons | Low-cardinality grouping fields; trace IDs needn't be indexed as dimensions | Avoids routine identity collection in error analytics | Adopt |
| Aggregate counters only | Strongest minimization and lowest event volume | Fixed dimensions and explicit windows | Smallest event-level exposure | Use when traces and exemplars aren't required |
This isn't a universal ranking. Aggregate counters are preferable for a mature, high-volume experiment whose error taxonomy is stable. Event-level telemetry earns its cost when engineers must inspect a small number of representative failures or correlate an error with a distributed trace. Stable pseudonyms belong in a controlled fraud or account-support workflow when repeated behavior is the actual subject; they are not suitable merely because an analyst might want an extra grouping key later.
Critical path: enforce the envelope before export
The critical path begins at instrumentation, not in a dashboard filter. Define an allowlist in the application or telemetry processor, normalize exception data, validate bounded values, and export over HTTP. This illustrative request uses a pseudonymous telemetry host and a verified error-capture route; it includes no direct identity. The trace and span values are synthetic.
curl --request POST \
--url https://telemetry.example/v1/errors/capture \
--header 'Content-Type: application/json' \
--data '{
"resourceLogs": [{
"resource": {
"attributes": [
{"key": "service.name", "value": {"stringValue": "workspace-api"}},
{"key": "service.version", "value": {"stringValue": "2026.08.16.3"}},
{"key": "deployment.environment.name", "value": {"stringValue": "production"}}
]
},
"scopeLogs": [{
"scope": {"name": "experiment-error-exporter"},
"logRecords": [{
"timeUnixNano": "1786838400000000000",
"severityText": "ERROR",
"body": {"stringValue": "dependency_timeout"},
"traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
"spanId": "00f067aa0ba902b7",
"attributes": [
{"key": "error.type", "value": {"stringValue": "DependencyTimeout"}},
{"key": "experiment.cohort", "value": {"stringValue": "variant_b"}}
]
}]
}]
}]
}'
Validation should fail closed for forbidden keys and fail visibly for malformed allowed values. A deployment pipeline can send a synthetic event for each cohort, confirm that the collector accepts the schema, and verify that the aggregate query returns one count per expected release-environment-cohort cell. A 429 response should trigger bounded retries with jitter and a local queue limit; dropping policy must prioritize exact counters over verbose exemplars. Don't let retry buffers become an unbounded secondary retention store.
Operational ownership matters as much as field selection. The experiment owner defines the cohort vocabulary and expected exposure count. The service owner maintains the error taxonomy. The privacy owner reviews linkability and deletion requirements. The observability owner measures bytes per accepted event, distinct values per indexed field, sampling rates, and retention by dataset. A release is ready only when those contracts agree.
Rejected option and the narrow case for it
The rejected design is “capture the full request, then redact in the observability backend.” It looks flexible because investigators can search headers, bodies, and user identifiers after an error. It fails this decision record because collection precedes classification, redaction rules lag application fields, high-cardinality values inflate storage, and every downstream copy expands the deletion surface. It also makes experiment analysis noisier: the query author must rediscover the cohort definition from incidental request data.
Keep full-request capture only for a narrowly authorized diagnostic system where the payload itself is the subject, consent or another valid basis has been established, access is restricted, retention is short, and ingestion performs field-aware filtering before persistence. Even there, don't reuse that dataset as the routine cohort scorecard. The experiment scorecard should divide error counts by cohort exposure, stratify by release and environment, preserve the sampling denominator, and state a minimum decision window before anyone reads a fluctuation as an effect.
This choice gives up ad hoc identity joins. The return is a dataset whose semantics match the decision: compare errors across tenant cohorts without retaining a shadow customer directory. When an investigation requires a person-level join, escalate to the purpose-built system and audit that access rather than quietly widening every error event.
Top comments (0)