Keep credential inventory, application audit evidence, and invoice usage as distinct records joined by stable identifiers. For a customer support service that meters customer usage, the deciding constraint is whether the service must refuse traffic before a customer's spending ceiling is exceeded. That decision needs an admission control record; an access review needs evidence of both granted authority and observed activity.
TL;DR: API key inventory answers which credentials have authority. Application audit logs answer which recorded actions occurred. Neither, on its own, establishes the complete chain from authorized customer request to billable usage. Preserve that chain explicitly, and spend the telemetry budget on evidence that can resolve a disputed invoice.
The architectural decision here concerns evidence provenance: what each record can establish, and where an inference must stop. OWASP's Secrets Management Cheat Sheet treats secret lifecycle management and auditing as related responsibilities. The design below applies that distinction to customer support metering; its numerical examples are stated assumptions, not production measurements.
How do API key inventory and application audit logs support an access review?
Consider a support application in which each customer has credentials for submitting work. A proposed inventory record contains a credential reference, customer identifier, owner, permitted scope, lifecycle status, and effective timestamps. This is the authority side of the review. A proposed audit event contains the credential reference, customer identifier, action, outcome, occurrence time, and a correlation identifier. This is the activity side. Store references to secrets in these records, never the secret values themselves; OWASP's guidance on auditing secrets supports excluding plaintext secrets from logs.
The distinction matters even when the two datasets agree. An active credential with no matching events could be unused, or its activity could fall outside the retained evidence. A current inventory snapshot also cannot establish what permissions existed when an older request arrived. The review therefore needs lifecycle history whenever its question reaches into the past. These are limits of the proposed evidence model, not reasons to collect every request payload.
Absence proves little.
For a metered invoice, add a third question: which completed work became a chargeable usage entry? A successful authorization event establishes permission under the proposed policy. It does not establish that work completed or that its usage was committed. A refused request should carry an auditable refusal outcome, but it should not acquire a completed usage entry merely because its attempt appears in a log.
This is also a practical review order. Identify the customer's credential set for the relevant period, join recorded actions through stable references, and then reconcile completed work against usage entries. Treat unmatched records as unresolved evidence. Do not silently convert them into zero usage.
Decision and invariants at the admission boundary
For this design, use an authoritative admission ledger for reservations and a separate usage ledger for completed, invoiceable work. Retain credential lifecycle history and audit the transitions that connect them. The application runtime does not change these evidence boundaries.
The first invariant is customer binding: the authenticated credential determines the customer before admission. A customer identifier supplied in a request body must agree with that binding; it is not sufficient authority by itself. The second is uniqueness: one logical work item has one stable admission identifier and at most one completed usage entry. Repeated submissions must resolve to that existing work item, rather than create new invoice usage.
The ceiling invariant is more demanding. Under a strict ceiling, admission must account for both committed usage and outstanding reservations in the same authoritative decision. Otherwise, two simultaneous requests can each see apparent headroom and consume it twice. In the illustrative arithmetic, a ceiling of 100 accounting units, 80 committed units, and 15 reserved units leaves 5 units available. A request needing a reservation of 6 must be refused under that policy. These are internal accounting units, not currency or a provider price.
Reservations require a defensible upper bound. If the work cannot be bounded before execution, this architecture cannot promise a strict ceiling solely through admission. The team must constrain work size, authorize additional reservations during execution, or explicitly accept an overrun policy. That choice belongs in the customer contract and the operational runbook.
Make refusal explicit.
For the strict policy proposed here, an unavailable authority store means refusing new metered work. This trades availability for ceiling enforcement. Already admitted work needs a defined completion and reconciliation path; deleting its reservation merely because a worker timed out would erase the distinction between abandoned work and work that is still running. Recovery must establish the work's state before releasing its capacity.
Compare the evidence each option preserves
The relevant comparison is between records and their proof obligations. None of these rows represents a commercial product.
| Evidence option | Useful answer | Boundary for a customer invoice |
|---|---|---|
| Current credential inventory | Which credentials currently have configured authority? | Cannot reconstruct earlier permission changes without history |
| Credential lifecycle history | Which authority changes were recorded over the review period? | Does not establish that authorized work ran |
| Application audit events | Which actions and outcomes were recorded? | Completeness depends on capture and retention; success needs a defined meaning |
| Admission and usage ledgers joined to audit evidence | Which work was admitted, completed, and counted? | Requires reconciliation of pending work and durable uniqueness rules |
The last option costs more engineering effort. Its main limitation is the operational burden of an authoritative ledger: this design is not suitable for a team that cannot reconcile pending work or accept refused traffic during admission-store unavailability. For a present-tense access review without metered billing, choose credential inventory and lifecycle evidence instead. That trade-off avoids adding an accounting system to answer an ownership question.
The ledger introduces state transitions, recovery decisions, and a schema that both billing and service owners must understand. Its benefit is a narrower inference: an invoice line can point to a committed usage entry, which points to admitted work, which points to the credential authority used at admission. That chain is the acceptance criterion for this design.
Critical path and the telemetry budget
The critical path is authenticate, bind the customer, reserve capacity, execute, and commit actual usage. Audit each consequential transition with the same admission reference. Reconcile any reservation left pending after execution. The following curl request illustrates a proposed internal admission contract at a pseudonymous endpoint; it is not an existing public API. Its credential is a placeholder. The contract requires a repeated idempotency key with the same payload to return the existing admission, and a changed payload with that key to be rejected.
curl --fail-with-body --silent --show-error \
--request POST \
--url 'https://metering.example.com/admissions' \
--header 'Authorization: Bearer REPLACE_WITH_TEST_CREDENTIAL' \
--header 'Content-Type: application/json' \
--header 'Idempotency-Key: support-work-0042' \
--data '{"customer_ref":"customer-17","work_ref":"support-work-0042","reserve_units":6}'
The receiving service must validate that requested reservation against its own work bound. A caller must not obtain admission by understating expected usage. The proposed response contract distinguishes admitted work from ceiling refusal, authorization refusal, and unavailable admission authority. Record those outcomes separately so a review can distinguish demand that was refused from work that became invoiceable.
At the observability layer, count cardinality before adding dimensions. In a hypothetical deployment with 200 customers, 4 operation classes, and 3 outcomes, their Cartesian product contains 2,400 possible combinations. Adding a credential dimension with 5 values per customer expands that design to 12,000 combinations. These are possible combinations, not measured active time series. Request identifiers would increase the dimension space further; keep them in evidence records rather than aggregate metric labels.
Retention has equally concrete arithmetic. Assume 1,000,000 audit events per day averaging 600 bytes each. That is 600,000,000 bytes per day and 18,000,000,000 bytes over 30 days, before indexing, replication, or compression. Keeping another field on every event has a repeated storage consequence. Define the invoice dispute and access review windows first, then choose retention for the records needed to answer those questions. Operational debug retention can be shorter if those records do not carry required evidence.
Sampling is a separate decision. In this design, usage commits, credential lifecycle changes, admission decisions, and reconciliation outcomes are required evidence and remain unsampled. Diagnostic traces may be sampled because their role is explanation rather than accounting completeness. Retaining every refusal helps investigate ceiling enforcement, but it also makes refusal volume part of the storage budget. Prefer a compact refusal schema over storing support message bodies.
The payload is unnecessary here.
Before deployment, exercise simultaneous requests near the ceiling, repeated admissions, changed payloads under the same idempotency key, and a worker stopping between completion and usage commitment. Check the resulting ledger and evidence chain, not merely the HTTP response. During rollout, reconcile admitted work against completed usage and pending reservations before treating a new schema as invoice evidence. Assign ownership of unresolved entries so reconciliation has a human destination.
Rejected option and its valid use case
Reject sampled application logs as the source of invoice usage or strict ceiling enforcement. Their selected evidence cannot establish complete usage, and a logging pipeline is not the authoritative reservation decision in this architecture. Reconstructing invoices from those logs would attach an accounting obligation to records deliberately retained for a different purpose.
Sampled logs remain useful for diagnosing slow support requests and explaining representative execution paths. A current credential inventory also remains useful for a present-tense ownership review. Both have legitimate, narrower jobs.
The resulting decision accepts additional ledger and reconciliation work to make refused traffic, granted authority, and invoiceable usage independently explainable. Preserve the identifiers and transitions needed for that explanation. Keep the remaining telemetry only as long as its operational question requires.
References
- OWASP Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
Top comments (0)