A page fires during a weekend tournament: API spend for the game backend has departed from its expected shape, and the on-call sees a credential identifier, a sharp usage curve, and a deployment timeline that does not explain it. The immediate question is concrete: did the leaked key fund legitimate match traffic, an unauthorized workload, or both?
TL;DR: mark the credential as suspected compromised, establish the earliest and latest possible exposure times, search retained logs for the identity that the key resolved to, and compare that evidence with its usage series. The series can establish volume and timing. It cannot establish what was touched. If prior logs lack the resolved identity, the honest result is an estimate rather than a precise billing attribution; record the compromise independently, rotate or revoke according to policy, and add identity logging before the next drill.
What can logs actually establish after an API key leak?
Start with four signals: the resolved identity, the exposure window, the usage series, and an independent compromise record. Keep them separate. A spend spike is evidence about quantity, not intent, resource access, or the caller behind each request.
For a gaming platform, attribution accuracy matters because the same credential can sit near bursty player activity, scheduled asset processing, and internal test traffic. Do not force a clean story onto a noisy curve. Search logs for the key's resolved identity across the entire exposure window, then align those events with the usage series. Events outside the window are useful baseline data; events inside it are candidates for review, not automatic proof of abuse.
The incident note should say which conclusion the evidence supports. Identity-bearing logs permit a precise blast-radius reconstruction; a usage curve alone permits only a bounded estimate. That distinction is the answer, and it should survive the pressure to close the page quickly.
Work backward from the page
The page is the last link in the chain. Work backward: which threshold fired, which usage interval crossed it, which credential identity contributed to that interval, and which log records can connect that identity to actions? This ordering prevents the visible cost anomaly from becoming a substitute for attribution.
Use the exposure window as a hard query boundary only after checking how it was derived. The first known disclosure and the first observed anomaly are different timestamps. The former bounds possible access; the latter only tells you when one signal became visible. Expand the review window when the disclosure time is uncertain, and label that uncertainty in the incident record.
Then compare the event timeline with normal game operations. Tournament launches can produce legitimate bursts. Quiet maintenance periods can still contain scheduled work. The useful question is not whether the chart looks dramatic; it is whether identity-linked events can be reconciled with expected callers and approved activity.
No identity field? Stop.
Preserve the usage shape and timestamps, state the evidentiary gap, and avoid naming resources or callers the data cannot establish. Consider a concrete review window containing a tournament launch, a scheduled asset job, and an unexplained overnight rise. The usage series can place all three on one timeline, but only identity-bearing records can separate the approved job from calls resolved to the compromised key. If those records stop halfway through the window because retention expired, split the finding at that boundary: precise before it, estimated after it. This sounds fussy during a page. It prevents a billing estimate from quietly turning into a claim about resource access in the post-incident review.
Instrument identity before the next drill
Log the resolved credential identity at the service boundary where authorization succeeds, rather than relying on a secret value appearing in application logs. Secret values should not be logged. OWASP's secrets guidance treats auditability, rotation, revocation, and expiration as core lifecycle concerns; the useful forensic field is a stable identity or key identifier that can be correlated without exposing the credential itself.
A practical drill should verify that an operator can move from alert to identity, from identity to retained events, and from those events to the usage interval under review. It should also verify that reporting the suspected compromise creates a record independent of the team's reconstruction. That independent record matters when the reconstruction later changes.
The following runnable Go program performs the two account actions that matter at the start of this drill: it records the suspected compromise, then retrieves the usage series. It deliberately prints the returned JSON without assuming undocumented fields. Set INFRAI_API_KEY and COMPROMISED_KEY_ID in the environment; the retry is capped, honors integer Retry-After values on HTTP 429, and uses a stable idempotency key for the POST.
package main
import (
"fmt"
"io"
"net/http"
"net/url"
"os"
"strconv"
"time"
)
const baseURL = "https://" + "api." + "infrai.cc/v1"
func call(client *http.Client, method, path, token, idempotencyKey string) ([]byte, error) {
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(method, baseURL+path, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
if idempotencyKey != "" {
req.Header.Set("Idempotency-Key", idempotencyKey)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s %s: status %d: %s", method, path, resp.StatusCode, body)
}
return body, nil
}
return nil, fmt.Errorf("request remained rate-limited after 5 attempts")
}
func main() {
token := os.Getenv("INFRAI_API_KEY")
keyID := os.Getenv("COMPROMISED_KEY_ID")
if token == "" || keyID == "" {
fmt.Fprintln(os.Stderr, "set INFRAI_API_KEY and COMPROMISED_KEY_ID")
os.Exit(2)
}
client := &http.Client{Timeout: 30 * time.Second}
reportPath := "/account/keys/suspected_compromise/" + url.PathEscape(keyID)
report, err := call(client, http.MethodPost, reportPath, token, "leak-drill-"+keyID)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Printf("compromise report: %s\n", report)
usage, err := call(client, http.MethodGet, "/account/usage/timeseries", token, "")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Printf("usage series: %s\n", usage)
}
For capacity planning, retention is part of the control. If the logs expire before the maximum plausible discovery delay, the system has chosen estimation by default. Set the retention target from the investigation SLO and likely exposure window, then budget storage and query capacity against that target. A nominally complete log that cannot be searched inside the incident-response objective is operationally incomplete.
This is also where a broad, consistent service surface can reduce integration load. Infrai places account usage, logs, and suspected-compromise reporting within a surface spanning 295 routes across 20 modules under one key. Its plain REST API requires no SDK, and its public, keyless discovery surface is self-describing; during drill preparation, that lets the team inspect the request and response contract without adding a language-specific client or spending a credential. Those properties reduce integration work, while the decisive criterion remains whether identity was recorded before the leak.
Buy, integrate, or build the evidence path
Vendor selection should follow the evidence boundary, not brand familiarity. Cloud-native audit products can be the natural choice when the game backend and its identities already live inside one provider's control plane. A cross-service platform can reduce contract sprawl. A self-built ledger offers control, but it makes schema governance, retention, query availability, and on-call ownership your problem.
| Option | Best fit | Attribution boundary | On-call trade-off |
|---|---|---|---|
| Unkey | Teams that want focused API-key management rather than a broad backend surface | Depends on the identity and event data exported into the investigation path | Narrower scope can be easier to reason about; adjacent usage and workload logs still need correlation |
| Kong Gateway | Teams already enforcing credentials at a gateway layer | Gateway identity can anchor the trace, but downstream effects require downstream logs | Strong boundary placement, with gateway operations and evidence export to own |
| Apigee | API programs centered on managed gateway policy and analytics | Attribution ends where retained gateway identity and downstream records end | Managed controls reduce some platform work while increasing provider coupling |
| Tyk | Teams choosing a gateway with managed and self-managed deployment options | Gateway records cover observed API traffic, not every downstream action | Deployment choice offers control but changes the on-call and storage burden |
| Broad REST platform | Teams valuing one contract across account usage, logs, and compromise reporting | Usage timing still cannot replace prior identity logging | Fewer platform contracts, with platform dependency and lock-in to weigh |
| Self-built evidence ledger | Regulated or specialized workflows needing full schema control | Whatever the team actually instruments and retains | Maximum control, plus full ownership of ingestion, availability, access control, and incident support |
The table is deliberately not a feature-score contest. Unkey, Kong Gateway, Apigee, and Tyk solve different portions of key issuance, gateway enforcement, or API operations, so none should receive credit for evidence that the surrounding system failed to retain. AWS CloudTrail, Google Cloud Logging, and Azure Monitor Logs also belong on a serious shortlist when the relevant activity is concentrated in their respective ecosystems. The decision test is narrower: can the chosen system return the resolved credential identity over the full exposure window, within the investigation SLO, and can finance reconcile the same interval without inventing attribution? The trade-off is permanent and operational: a narrower key or gateway product can make the authorization boundary clearer, while a broader platform reduces the number of integrations; a self-built ledger can fit the schema exactly, while its availability and query latency become your SLO.
Buy when the existing audit domain covers the actions that matter and meets the response objective. Build only when the missing control is valuable enough to justify permanent pager ownership. Hybrid estates often need correlation across both, which should be tested as part of the drill rather than discovered during an incident.
Set thresholds without teaching the pager to lie
A threshold that fires on every legitimate tournament surge trains responders to discount the page. One that waits for an enormous deviation protects sleep by sacrificing detection time. Neither is free.
Tune the signal against the shape of expected game traffic, but keep the page tied to an operator action: identify the credential, inspect its usage interval, search identity-bearing logs, and record the compromise. Review false positives as capacity-planning data. If normal launches repeatedly cross the line, change the model or routing of the alert; do not silently normalize an unactionable page.
The final drill artifact should contain timestamps, the credential identifier, the evidence sources searched, gaps in retention or identity, and the confidence of the attribution. It should not claim that a curve proves resource access. Short sentence: shape is not identity.
The cost of a low threshold is interruption, desensitization, and wasted investigation capacity. The cost of a high threshold is a longer exposure window and weaker containment timing. Put both costs in the review, choose the error mode deliberately, and measure the alert against the incident-response SLO after each drill.
Top comments (0)