DEV Community

nRouter
nRouter

Posted on

Log the metadata, not the message

Request logs are the first thing you reach for when something breaks and the last thing you want leaked. They are your best debugging tool and a concentrated store of exactly the data you told customers you would protect. The resolution is not a philosophy. It is a field list.

Split the log in two: metadata and content.

Metadata: log it generously

  • Identity — request ID, key, organization. Which call was this, and whose?
  • Shape — model, provider, prompt / completion / total tokens.
  • Outcome — status, latency, cache-hit status.
  • Money — the settled cost of that one call.
  • Time — start and end timestamps.

Cost by model, latency percentiles, error regressions, cache effectiveness, per-key spend — all answerable from that list, without a single character of prompt text. The standards agree: OpenTelemetry's GenAI span convention requires the operation and provider names, recommends the model, finish reasons, response ID and usage token pair, and treats message content as opt-in because it is likely to contain sensitive information.

Content: a decision, not a default

The prompt and the completion buy you a narrow slice of extra value — reproducing one specific bug, sampling for quality review — and nearly all of the risk. On the gateway in the guide below, content is never written to the log; metadata always is, because metadata is the billing record. The data policy changes what the dashboard shows, not what is stored, and it is an admin-only lever. Full-content logging is a roadmap item, and the honest interim rule is: if you need the exact prompt weeks later, capture it on your own side, deliberately, with your own redaction.

Three disciplines that make the log trustworthy

  1. Capture the request ID. Every response carries it in a header. Store it beside your own trace ID on the success path and the exception path. Skip this and every investigation degrades to searching by timestamp.
  2. Absent is not zero. When a call cannot be priced, the cost header is omitted and a status header says unpriced. Default the missing value to zero in your client and your spend dashboard drifts from the ledger forever, silently.
  3. Redact before you copy. CSV exports, forwarded headers, a second observability stack — your copy can carry content too. Redaction after the copy is theatre.

And keep the categories apart: who changed a budget or rotated a key belongs to the audit trail, a separate store with different volume and retention. Retention itself is a policy with an owner — bounded windows, exports for what must outlive them. Keep aggregates long, raw rows short, content shortest of all.

The full guide — field table, the four-step debugging workflow, the gateway-logs-versus-self-hosted-traces comparison, and a five-step verification routine — is here: https://nrouter.ai/blog/guides/llm-observability-request-logs

Top comments (0)