Short answer: Keep a small, current inventory of tenant-scoped keys and a separately retained record of what each key did. Inventory answers which credentials can access a customer-support tenant now; application audit events answer which credential acted, when, and on whose behalf. The cheapest defensible design is usually to retain every credential lifecycle event but limit high-volume request records to the fields and period your review actually needs. A revoked key disappearing from the active list must not erase its history.
What actually fills the bill?
Start with event volume, not key count. Suppose a support integration owns 200 active tenant keys and sends 10,000 requests per day. At one request event per call, that is 300,000 events over a 30-day window, while the active inventory still has roughly 200 rows. Those are illustrative inputs, not a measured workload. A serialized event averaging 1 KB would yield about 300 MB of raw events before indexes, replication, and backups; recording response bodies or repeating full tenant metadata makes the dominant term larger. The inventory is rarely the storage problem. Retention and queryable event volume are.
The tempting change is to log only failures. It cuts volume, but it removes the successful reads an investigator needs after a support agent reports an unexpected customer-data view. Record a compact event for each decision instead: timestamp, tenant ID, stable credential ID, actor or workload identity, action, resource category or identifier, decision, and request correlation ID. Do not log the secret itself, OTP values, message bodies, or customer conversation content. Those fields create another disclosure surface without answering the access question.
Bodies are out.
What can API key inventory answer that application audit logs cannot?
For a tenant access review, inventory is a snapshot: key ID, tenant binding, allowed scopes, issuing principal, creation time, status, and revocation time. It lets an owner find an overbroad key and revoke it. A request audit event is historical evidence. It must preserve the credential ID even after revocation, so the reviewer can ask whether that same key read tickets before the status changed.
There is a sharp boundary here. A successful authentication event does not prove an authorized ticket read; an application action event should record the authorization decision at the point of use. Conversely, a key marked revoked today does not prove that it never accessed another tenant yesterday. The reviewer needs both datasets and an explicit join on a stable, nonsecret identifier. If a key can be reissued under the same display name, names are not identifiers.
That join must survive deletion from the active view.
How do you keep revocation auditable?
Persist issuance, scope changes, and revocation as append-only lifecycle events alongside the current inventory. In the request path, resolve the presented secret to its credential ID, check status and tenant binding, enforce scope, and emit an action event with the resulting decision. Reject a request whose tenant does not match the key even if it carries a valid signature or token. That last condition matters in shared support systems: a perfectly valid credential for tenant A must not authorize tenant B.
One way to check the evidence contract without relying on a particular storage backend is to join a test fixture by immutable credential ID. The function deliberately reports missing records rather than treating absence as proof that no access occurred:
def review_key(key_id, inventory, lifecycle, actions):
key = next((row for row in inventory if row["key_id"] == key_id), None)
changes = [row for row in lifecycle if row["key_id"] == key_id]
uses = [row for row in actions if row["key_id"] == key_id]
return {"current_key": key, "changes": changes, "uses": uses,
"history_missing": not changes}
An empty uses list means only that this query returned no events within its available retention window. It cannot establish that the key was never used. Test that distinction with an expired event window and a lifecycle record that still identifies the revoked key.
Treat delivery of an audit event as a design decision. If the audit sink is temporarily unavailable, decide in advance which sensitive actions must fail closed and which may proceed through a durable local queue. A silent best-effort write is not evidence. Queue replay should preserve the original event time and event ID so retries do not masquerade as new activity. Protect the event store from application operators who can issue keys; otherwise the same principal can grant access and alter its trail.
Here is a useful test: issue a key for one support tenant, perform one permitted ticket read and one cross-tenant denied read, then revoke it and retry the permitted read. Expect three action events with the same credential ID, distinct decisions, and the correct tenant identifiers, plus lifecycle records for issue and revoke. Query the history after the active inventory no longer lists the key. If the join breaks, the access review breaks.
What should you stop retaining?
Set a retention period from the organization's incident-response and legal requirements, then make deletion explicit and testable. Keep lifecycle evidence for the chosen review window; retain compact action events for the period in which investigations may need per-request detail. After that window, retain aggregates only if they serve an identified operational question. Aggregate counts can show that a credential was busy, but cannot reconstruct a particular ticket read. That is the cost of discarding detail, and reviewers should agree to it before an incident.
The limitation of compact events is that a resource category alone may be too coarse to identify the affected ticket. Retaining exact ticket IDs improves investigation but expands the sensitive metadata footprint, especially where ticket identifiers can be linked to customer records. Choose the minimum identifier resolution needed for a real review, restrict readers of the event store, and document the loss of detail at expiry. A short retention window is a poor fit when investigations routinely begin after it closes; extending the window costs storage and increases the amount of sensitive history to protect. Neither choice fixes a missing authorization decision in the application path.
No replay can restore discarded detail.
Revisit the calculation using real event sizes, request rates, indexes, and backup copies before changing policy. During rollout, check for missing tenant IDs, duplicate replayed event IDs, clock skew, and gaps between authorization decisions and persisted events. Sample an access review with security and support staff; a schema that cannot answer their concrete question is expensive even when storage is cheap.
Further reading
- OWASP Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- OWASP Logging Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
- NIST SP 800-53 Rev. 5, Audit and Accountability controls: https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final
Top comments (0)