DEV Community

JamesAnderson121
JamesAnderson121

Posted on

Metered Invoice Tooling: Separate Console Credentials and Least-Privilege Access

#security #api #billing #architecture

The constraint that decides this isn't the threat model. It's the dispute email that lands six weeks after the invoice went out: a merchant on the e-commerce platform says the metered line for AI-generated product descriptions is wrong, and someone has to show which principal adjusted that usage record. A shared production credential cannot answer that. Use a second credential that belongs to the internal admin console alone — scoped to the handful of endpoints support staff actually touch — and carry the operator's identity in a short-lived token layered on top of it.

Two credentials, two different jobs. One says which program is calling. The other says which person asked it to.

Why a shared production credential can't answer "who changed this line?"

Metered billing has a shape that makes attribution load-bearing. Usage events land in a ledger, an aggregation job rolls them into invoice lines, and the invoice becomes a financial document you may have to defend to a merchant, an auditor, or a payment processor running a chargeback. The console — a small Node.js service sitting behind SSO — lets support re-run an aggregation window, void a line, or write a manual correction with a reason code. Each of those is a mutation against the same billing API that the production workers call all night. Authenticate the console with the production key and the API's own access log records one identical key id for the nightly aggregation worker and for a support engineer's 3 a.m. correction. At the layer that actually gets subpoenaed, the two events are indistinguishable.

Writing your own audit table inside the console is the obvious first move. It helps, and it's weaker than it looks: that row is written by the same process whose behavior is in question, so it records the application's intent rather than the platform's observation. When an internal log and an API log disagree, the one the merchant's lawyer trusts is the one the application couldn't edit.

OWASP's secrets guidance is blunt about the boundary. Secrets should be scoped to a single consumer and a single environment, rotated independently, and never shared across a trust boundary. An internal tool driven by humans and a batch worker driven by a scheduler sit on opposite sides of that line, even when both call the same endpoint.

Should the admin console have its own API key, separate from the production credential?

Yes — with one correction to the way it usually gets built. The console's credential should identify the tool, not the person. Per-person API keys sound like least privilege and drift into the opposite: they get pasted into browser storage or a shared vault entry, nobody revokes them after an offboarding, and their scopes ratchet upward every time someone hits a 403 mid-incident.

Split the two facts a request needs to establish.

The workload credential establishes that the caller is the admin console, deployed in the internal network. It's provisioned once, stored in the deployment's secret manager under its own name, rotated on a schedule, and granted a narrow allowlist: read one customer's usage ledger, re-run one aggregation window, write a correction. Nothing else. It cannot mint credentials, cannot bulk-export other merchants, cannot reach payout endpoints. When that key leaks, the blast radius is the set of things support was already allowed to do — which is the whole point of separating it from production.

The actor token establishes that an authenticated employee asked for it. RFC 8693 covers the mechanism: the console presents the operator's SSO token together with its own credential and receives back a token whose sub is the human and whose act claim is the console. Delegation becomes explicit and verifiable server-side instead of implied. TTL measured in minutes, not days.

The header that proves nothing, and what replaced it

The first version of this used one production key plus an X-Acting-User header set by the console. It reads fine in a code review. Under dispute it's worth nothing, because the header is an unauthenticated assertion — anyone holding that key can send any value in it, including a service that isn't the console at all. An audit trail whose operator field can be forged by every holder of the credential is decoration.

The replacement records five things the console cannot fabricate on its own:

Recorded field Comes from Question it answers
key id console workload credential which program wrote this
sub exchanged operator token which human asked for it
act token exchange which tool acted on their behalf
granted scopes the credential's allowlist what it was permitted to do
request id the gateway which HTTP call, end to end

Append-only storage for that stream is not optional. If the same operators who can void an invoice line can also rewrite the record of voiding it, the design collapses back to trusting the application.

The test harness that keeps this honest

Attribution is a property you verify, not a property you assume. This one started as a notebook cell and belongs in CI: pull the last day of audit events, keep the mutating ones, and assert every single one names both a tool and a person who still works here.

import os
import time
import requests

AUDIT = os.environ["AUDIT_API"]            # read-only replica, its own credential
TOKEN = os.environ["AUDIT_READ_TOKEN"]     # read scope only, no billing writes
MUTATIONS = {"usage.correction.write", "invoice.line.void", "aggregation.rerun"}


def recent_events(minutes=1440):
    r = requests.get(
        f"{AUDIT}/internal/audit/events",
        params={"since": int(time.time()) - minutes * 60},
        headers={"Authorization": f"Bearer {TOKEN}"},
        timeout=10,
    )
    r.raise_for_status()
    return r.json()["events"]


def unattributed(events, active_people):
    for e in events:
        if e["action"] not in MUTATIONS:
            continue
        actor = e.get("actor", {})
        # a console mutation must name the tool (act) and a current employee (sub)
        if actor.get("act") != "admin-console" or actor.get("sub") not in active_people:
            yield e


if __name__ == "__main__":
    people = set(requests.get(
        f"{AUDIT}/internal/directory/active",
        headers={"Authorization": f"Bearer {TOKEN}"},
        timeout=10,
    ).json())
    gaps = list(unattributed(recent_events(), people))
    print(f"unattributed mutations: {len(gaps)}")
    raise SystemExit(1 if gaps else 0)
Enter fullscreen mode Exit fullscreen mode

The target is zero, and a non-zero result is informative either way: either the delegation path has a hole, or some other service quietly borrowed the console's credential. Three more numbers are worth watching before you copy this design. How long it takes to answer "who changed invoice 4417" — it should be one query against one stream, not a grep across three systems. How many scopes the console credential carries this quarter versus last, since scope count only ever goes up unless someone measures it. And rotation lag, because a separate credential that nobody has rotated since launch is just the production key with extra steps.

When a single shared key is still the right call

If one engineer runs the whole billing stack, the actor-token layer costs more ceremony than it returns; a personal token, a short TTL and an honest shell history are a defensible answer at that size. Stick with the simpler setup until more than one person can touch an invoice — somewhere past that point the argument flips hard, though I'm not sure the boundary is as crisp as a headcount number makes it sound.

Token exchange also isn't universal. If your identity provider doesn't support it, per-operator tokens issued by your own service with a 12-hour lifetime and a documented break-glass path get you most of the attribution benefit. The trade-off is real: putting the identity provider on the console's critical path means an authentication problem becomes a support-tooling problem at the worst possible moment, so keep a sealed emergency credential with alerting on every use.

And none of this fixes a metering pipeline that loses events. Attribution tells you who touched a number. Whether the number was right in the first place is a different article — and a different set of tests.

References

Top comments (0)