A leaked credential may still be serving production traffic while responders investigate it. The operational constraint is to preserve service availability without losing the ability to distinguish old-key traffic from new-key traffic. Short answer: issue a replacement, move legitimate callers to it, retain separate key identifiers in access records, and revoke the exposed key once the migration is verified. Then use request-level evidence to bound what the old credential actually accessed. A usage graph can signal activity; it cannot, by itself, identify the resources returned.
This is an incident-response experiment, not a throughput benchmark. The evaluation constraint is evidence quality: can another engineer reproduce the same set of requests, callers, resources, and remaining unknowns from retained records? A quick revoke may stop further use, but it can also interrupt callers that have not switched. Keeping the old key indefinitely avoids that interruption while extending exposure. The useful middle ground is a time-boxed overlap with explicit attribution for both credentials.
After an API key leak, how can logs establish what it actually touched?
Start by separating three claims: a request presented the leaked key, the service accepted that request, and a protected resource was returned. These are different events. A request count or token-usage series is valuable for finding an unusual interval, especially around the first known exposure, but it does not prove which document or tenant was touched. A successful authentication record alone does not prove the downstream operation succeeded either. If the gateway says accepted but the application has no matching request, check whether sampling, asynchronous processing, or a retention boundary accounts for the gap before labeling the request harmless. If the application records a resource read but no response outcome, distinguish attempted access from confirmed delivery; neither field can substitute for the other. Write down the exact evidence cutoff so the next responder does not silently extend a claim beyond the available records.
Counts aren't contents.
For a developer-tools API that also serves a Python retrieval workflow, the audit record needs a stable key identifier, event time, request or trace identifier, authenticated principal, operation, resource identifier, authorization result, and outcome. Keep the secret value out of logs. OWASP's secrets-management guidance calls for logging access to secrets while avoiding disclosure of the secrets themselves; its logging guidance also treats correlation identifiers and sufficient event context as useful for investigations. The key identifier here is an internal lookup handle, not a copy of the credential.
If the application logs only aggregate model tokens per minute, the retrieval side remains a blind spot: token counts cannot tell you which document was fetched before a model call. Join gateway authentication events to application authorization and resource-access events on the same request or trace identifier. Record a missing join as missing evidence, not as a clean bill of health. This matters when a notebook prototype becomes a production agent: its service key may be shared across runs, while the actual question is which run obtained which resource.
A focused reconstruction
Suppose the investigation window contains three requests using the exposed key identifier. One is denied at authorization, one completes a document read, and one has an accepted gateway event but no matching application outcome. The defensible blast-radius statement is one confirmed read, one denied attempt, and one unresolved request. Three requests do not equal three disclosed documents.
That third request matters.
The following Python sketch groups already-collected, redacted event dictionaries by request ID. It intentionally leaves unresolved requests visible instead of turning missing application logs into success or failure:
from collections import defaultdict
def classify_requests(events, exposed_key_id):
by_request = defaultdict(list)
for event in events:
if event.get("key_id") == exposed_key_id:
by_request[event["request_id"]].append(event)
findings = []
for request_id, records in by_request.items():
if any(r.get("stage") == "resource" and r.get("outcome") == "returned"
for r in records):
status = "confirmed_resource_return"
elif any(r.get("stage") == "authorization" and r.get("outcome") == "denied"
for r in records):
status = "denied"
else:
status = "unresolved"
findings.append({"request_id": request_id, "status": status})
return findings
Real records can contain multiple resource events per request; preserve each resource identifier and its authorization decision when exporting evidence. The sketch only demonstrates why a gateway-only row should stay unresolved. Check timestamps in a consistent time basis and document log-retention gaps before declaring the window complete.
Rotate without erasing attribution
Create a distinct replacement credential with the same necessary permissions, distribute it through the normal secret-delivery mechanism, and switch callers in a controlled rollout. Observe successful requests under the new key identifier while tracking old-key requests by caller and operation. Only then disable the exposed key; investigate any old-key traffic that persists after the intended cutover. This sequence does not make overlap risk-free. It makes the trade-off inspectable.
For a retrieval or agent service, include scheduled jobs, evaluation runners, and long-lived workers in the inventory. An eval run that still uses the old key can look like an attacker in a coarse usage series; conversely, an attacker can blend into aggregate token volume. Separate identities for workloads make the comparison more useful, while explicit per-request authorization and outcome records establish what happened. Keep the incident export access-controlled because resource identifiers and request metadata can themselves be sensitive.
What should be measured before adopting this method?
Run a rehearsal with a test credential and known requests. Measure the time until all legitimate callers stop using the old key, the fraction of gateway events that join to application outcomes, and the number of unresolved requests after the retention window is checked. Also measure whether a denied request, a returned resource, and an application error remain distinguishable in the export. No invented percentage is a substitute for that check.
There is a prompt-cost consequence too: investigating by replaying an agent's workload can incur fresh model calls and may expose different documents than the original request. Prefer retained event evidence for the incident boundary, and use an isolated eval harness to test whether instrumentation captures the next run correctly. If the joins or retention are incomplete, report the observed lower bound and the uncertainty rather than claiming the entire blast radius is known.
Top comments (0)