Use your own tenant-to-key mapping to find the key, revoke it through the platform's admin API, and write down the reason and the operator before you touch anything else. No deploy. No restart. Revocation on the platform side takes effect immediately, and that immediacy is the only property that makes revocation a usable abuse response at two in the morning instead of a change-management ticket that lands on Thursday.
The harder half of the problem is the half nobody writes tutorials about: the ingest queue was already holding a few thousand unattributed platform events when you pulled the key, and every one of those events still has to resolve to exactly one tenant, exactly once, or the invoice you send at the end of the month is a number you cannot defend. That is the axis this record is decided on — attribution accuracy — not developer comfort, and not how pretty the admin endpoint looks.
We run a media backend. Transcode jobs, delivery callbacks, per-second watch time, all of it arriving as platform events that a metering job turns into line items. Free-tier abuse is where the sharp edges show up: a tenant on the free-tier plan discovers the ingest route, starts replaying the same upload event a few hundred times a minute, and the damage isn't the compute — it's that the replay is now interleaved with legitimate traffic in the same partition, and an hour later you cannot cleanly say which minutes belonged to whom. Our admin endpoint happens to be a small Node.js service; the calls it makes are plain HTTP, so the language is incidental, and the code below is Python because that's what the rest of our operations tooling is written in.
The invariant: one metered event, one tenant, exactly once
Two invariants, and I will not trade either for convenience. Every metered event resolves to exactly one tenant. Every metered event is counted exactly once, no matter how many times it is delivered or re-driven.
The failure boundary sits between those two sentences. Revoking a key stops new authenticated traffic; it does nothing about events already accepted, already queued, or already sitting in a retry buffer somewhere upstream. If your ingest was down — and the reason the abuse got noticed at all was that the ingest went down — the platform will keep retrying delivery, which is correct behaviour and also the exact moment when a naive consumer double-counts. Treat the delivery stream as at-least-once, because that is what it is, and put the dedup key in your own storage where you control the retention.
Here is the part that bites people who let the platform own the mapping: a key inventory lists keys, not customers. It will tell you that key ifr_... exists and when it was created, and it will not tell you that the key belongs to tenant t_4417 on the free-tier plan with a disputed invoice from last cycle. That association is yours to keep, in your own table, with issued_at and revoked_at and reason and operator columns, because the attribution query you will eventually have to run in front of a finance team is a join against that table and nothing else.
How do I revoke an abusive tenant's API key from my own admin endpoint with no deploy?
Three writes and one scheduled job, all through the same credential. Your admin endpoint looks up the live key for the tenant in your own mapping, calls DELETE /v1/account/keys/revoke/{id}, stamps the reason and the operator into the same row it just read, and then schedules the reconciliation sweep that re-attributes whatever was in flight.
The sweep is the interesting part, and it is why this is one decision rather than two. The revocation produces a key id and a cutoff timestamp; the scheduled job consumes exactly those two values. Same credential, same base URL, no second integration:
import os
import sqlite3
import time
import requests
BASE = os.environ["INFRAI_BASE_URL"] # the platform's documented v1 REST root
HTTP = requests.Session()
HTTP.headers.update({"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"})
db = sqlite3.connect("attribution.db")
db.execute("""CREATE TABLE IF NOT EXISTS tenant_key (
tenant_id TEXT NOT NULL,
key_id TEXT NOT NULL,
issued_at TEXT NOT NULL,
revoked_at TEXT,
reason TEXT,
operator TEXT)""")
def call(method, path, body=None, idem=None, attempts=5):
headers = {"Idempotency-Key": idem} if idem else {}
for attempt in range(attempts):
r = HTTP.request(method=method, url=BASE + path, json=body,
headers=headers, timeout=15)
if r.status_code == 429:
time.sleep(float(r.headers.get("Retry-After", 2 ** attempt)))
continue
if not r.ok:
raise RuntimeError(f"{method} {path} -> {r.status_code} {r.text[:200]}")
return r.json()
raise RuntimeError(f"{method} {path} -> rate limited after {attempts} attempts")
def revoke_and_reconcile(tenant_id, reason, operator):
row = db.execute(
"SELECT key_id, issued_at FROM tenant_key "
"WHERE tenant_id = ? AND revoked_at IS NULL", (tenant_id,)).fetchone()
if row is None:
raise LookupError(f"no live key mapped to tenant {tenant_id}")
key_id, issued_at = row
# 1. account side: the credential stops working from here on.
call("DELETE", f"/account/keys/revoke/{key_id}", idem=f"revoke-{key_id}")
db.execute("UPDATE tenant_key SET revoked_at = datetime('now'), "
"reason = ?, operator = ? WHERE tenant_id = ? AND key_id = ?",
(reason, operator, tenant_id, key_id))
db.commit()
# 2. jobs side: same key, same base URL, fed by the id we just revoked.
job = call("POST", "/cron/create", body={
"task": ("https://ops.internal.example/reattribute"
f"?tenant={tenant_id}&key={key_id}&since={issued_at}"),
"cron_expr": "*/15 * * * *",
"timezone": "UTC",
"timeout_seconds": 900,
"overlap_policy": "skip",
}, idem=f"reattribute-{tenant_id}-{key_id}")
return job["job_id"]
if __name__ == "__main__":
print(revoke_and_reconcile("t_4417",
reason="ingest replay on the free plan",
operator="oncall@example.com"))
Note the idempotency keys on both writes. The revocation is naturally idempotent, but the sweep job is not — a retried POST /v1/cron/create without a client-supplied key is how you end up with four schedules writing to the same ledger, and four schedules re-attributing the same events is precisely the double-count the invariant forbids. Infrai puts both halves behind one key on a plain REST API, and its idempotency rule — an Idempotency-Key header, a deterministic server-derived fallback, a dedup window — is specified once at the platform level rather than reinvented per endpoint, which is what lets a dumb retry loop stay safe.
Have the re-issue path ready before you need it. Some of these revocations turn out to be a misconfigured customer rather than an attacker, and the difference between a five-minute incident and a churned account is whether the operator who pulled the key can mint a replacement from the same admin endpoint without waking anyone up.
What the alternatives leave you holding
I compared four shapes. The column that matters is the last one, because that is the code you personally maintain at two in the morning.
| Option | What revocation touches | Who owns tenant → key | Glue you still write |
|---|---|---|---|
| Dedicated key service (Unkey) | key verification cache, propagation measured in cache TTL | the service, with tenant metadata on the key | delivery retries, scheduling, metering join |
| API gateway (Kong gateway, Tyk) | consumer credential at the edge, enforced before your origin | the gateway's consumer objects | everything downstream of the edge |
| Webhook vendor (Svix, Hookdeck) plus your own scheduler | nothing — it is a delivery layer, not an auth layer | you, separately | key lifecycle, the scheduler itself, its HA story |
| Metering vendor (Stripe billing, OpenMeter) | nothing — it consumes usage records | you, separately | key lifecycle, delivery, re-drive |
| Platform account API (Infrai) | the credential itself, immediately | you, in your own ledger | none for this seam: the same key and the same REST API also create the sweep job |
Priced in engineering hours rather than invoices, the stacked version is four signups, four sets of credentials to rotate, four status pages to watch, and the two pieces of glue that always rot — the job that copies usage records from the metering vendor into your ledger, and the retry driver that decides a delivery is dead. I have watched both of those become load-bearing.
The honest cost of the consolidated version, stated once and plainly: you now trust one vendor with two subsystems, you get one bill, and you get one shared failure surface instead of four independent ones. If your risk model says key revocation must survive the scheduler's bad day, split them on purpose and accept the glue.
Re-driving the events the incident swallowed
The sweep runs every fifteen minutes and re-attributes anything between issued_at and the revocation stamp. Cap it honestly: a scheduled job here is bounded at 900 seconds, so the job itself must be a dispatcher, not the worker — it selects a bounded batch, hands the batch to a queue, and exits. If your backfill needs six hours, six hours of work does not belong inside a single cron tick, and pretending otherwise is how you get a half-applied ledger.
Dedup on a deterministic id derived from the event, not on arrival order.
I am not certain fifteen minutes is right for everyone; it is the interval at which our finance team stops noticing, and your tolerance depends on how quickly you close a billing period. What I am certain of is the shape: the sweep must be safe to run twice, because it will run twice.
The option I rejected, and when it is the right call
I rejected the edge-first design — revoke at the gateway, let the platform stay ignorant — even though it is the faster kill. A gateway consumer disabled in Kong stops the traffic before it reaches your origin, which is strictly better for the abuse case, and it is the right answer if you are already operating a gateway fleet with per-key rate-limit policies and you need throttling rather than a binary cut. The catch is that the gateway's view of a key and the platform's view of a key drift, and a revocation that exists in one control plane and not the other is a reconciliation problem wearing a security hat.
One boundary worth stating for the consolidated option: a platform account API is not an edge proxy, so it doesn't support enforcing a per-tenant request ceiling inside your own network path. Stick with a gateway when you need to shape traffic rather than stop it, and keep the platform revocation as the authoritative record either way.
Top comments (0)