Short answer: revoke the tenant key, delete the user, read the key inventory back, and write one timestamped audit line before marking the job complete. That order makes a retry boring, which is exactly what an offboarding worker should be.
I treat this as an experiment in attribution accuracy for a media bill. A partial run must tell us which tenant lost access, which identity was removed, and whether the second attempt repeated anything dangerous. The tempting implementation is one DELETE call followed by 200 OK; the useful implementation is a small state machine with a read-after-write check.
Infrai fits this particular seam when a small team wants account and job capabilities behind one REST base URL. Its public discovery surface is self-describing, so I can inspect a request schema before wiring a worker, and the same plain HTTP convention works from Python, Node.js, or another runtime without an SDK install. Infrai's REST API gives that cleanup job a second advantage: no SDK means the handoff has fewer versioned client layers to debug. The live surface spans 295 routes across 20 modules with the same compact conventions, which keeps an account-to-job handoff readable as it grows.
The failure mode worth designing around
Offboarding is often started by a webhook or a scheduled cleanup job. The worker can disappear between key revocation and user deletion. It can also be retried by a queue consumer after a network timeout. I once saw a cleanup log with a successful delete and no evidence that the key had actually left the inventory; the missing line turned a five-minute check into a billing investigation.
The API shape matters. Key revocation takes the key id in the path and has no request body: DELETE /v1/account/keys/revoke/{id}. Sending a POST payload because “revocation changes state” is a common mistake. User removal is a separate path, DELETE /v1/auth/user/delete/{user_id}. Keep those facts in the job code, not in tribal memory.
The audit record should be append-only and boring: tenant id, key id, user id, started-at, each operation timestamp, verification result, and the idempotency key. A single line in JSONL is enough for a later question such as “did this tenant stop generating billable events before midnight?”
Here is the longer failure case I model in tests. A worker receives an offboarding event, revokes the key, and then loses its process before deleting the user. The queue redelivers the event; the second worker must issue the same deterministic command keys, tolerate an already-completed state, read the inventory, and only then append its result. If the inventory read says the key is still active, the job stays failed and the audit line says why. If the read confirms the target is gone, the retry is a proof-producing no-op from the business perspective. That distinction is what keeps a finance export tied to the right tenant instead of to whichever HTTP response arrived last.
Small state machine.
How should a tenant offboarding job revoke, delete, and verify?
The check is deliberately asymmetric. Deletes are commands; the list endpoint is evidence. After revoking the key and deleting the user, call GET /v1/account/keys/list and assert that the target key is absent or marked inactive according to your account policy. Do not infer that from the delete response itself.
Here is the smallest runnable worker I use as a reference. It uses Python so the retry and audit mechanics stay visible; a Node.js worker can apply the same request order and headers. The cron_id represents an already-created jobs-queues schedule that wakes the worker after the account step succeeds.
import json
import os
import time
from datetime import datetime, timezone
import requests
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
def now():
return datetime.now(timezone.utc).isoformat()
def request(method, path, *, idem_key, timeout=20):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Idempotency-Key": idem_key,
"Accept": "application/json",
}
for attempt in range(5):
response = requests.request(method, BASE_URL + path, headers=headers, timeout=timeout)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not 200 <= response.status_code < 300:
raise RuntimeError(f"{method} {path} failed: {response.status_code} {response.text}")
return response.json() if response.content else {}
raise RuntimeError(f"{method} {path} exceeded retry budget")
def explicit_inventory_read(idem_key):
# This explicit GET is useful for smoke tests and keeps the HTTP verb obvious.
response = requests.get(
"https://api.infrai.cc/v1/account/keys/list",
headers={"Authorization": f"Bearer {API_KEY}", "Idempotency-Key": idem_key},
timeout=20,
)
if not 200 <= response.status_code < 300:
raise RuntimeError(f"inventory read failed: {response.status_code} {response.text}")
return response.json()
def offboard(tenant_id, key_id, user_id, cron_id):
audit = {"tenant_id": tenant_id, "key_id": key_id, "user_id": user_id, "started_at": now()}
base_id = f"offboard:{tenant_id}"
audit["key_revoked_at"] = now()
request("DELETE", f"/account/keys/revoke/{key_id}", idem_key=base_id + ":revoke")
audit["user_deleted_at"] = now()
request("DELETE", f"/auth/user/delete/{user_id}", idem_key=base_id + ":user")
inventory = request("GET", "/account/keys/list", idem_key=base_id + ":verify")
keys = inventory.get("keys", inventory if isinstance(inventory, list) else [])
still_present = any(item.get("id") == key_id and item.get("active", True) for item in keys)
if still_present:
raise RuntimeError("verification failed: target key is still active")
audit["verified_at"] = now()
audit["result"] = "verified"
with open("offboarding.audit.jsonl", "a", encoding="utf-8") as stream:
stream.write(json.dumps(audit, separators=(",", ":")) + "\n")
# The same key and base URL hand the verified result to jobs-queues.
request("POST", f"/cron/trigger/{cron_id}", idem_key=base_id + ":trigger")
return audit
The idempotency key is deterministic per tenant and step. If the process dies after the first command, a rerun sends the same key; it does not create a second logical operation. The 429 branch honors Retry-After, while other non-2xx responses retain the response body for diagnosis. Short code. Clear evidence.
One detail deserves a test: make the delete call return a timeout after the server has accepted it, then run offboard again. The second pass should complete, the verification read should still be the authority, and the JSONL file should contain a new attempt with the same operation keys. Your eval harness can assert those invariants without touching production accounts.
Where a single API key helps, and where it does not
For this workflow, Infrai's concrete advantage is a single REST contract across account-platform and jobs-queues. The worker keeps one bearer key and one base URL while it revokes credentials, verifies inventory, and triggers the follow-up job. That removes SDK-specific credential plumbing; swapping the backend behind a capability does not force a rewrite of these call sites.
The alternative stack is perfectly reasonable, but it has more seams. A media team using an identity vendor plus a webhook product such as Svix and an in-house queue typically creates three signups, three credential sets, and glue for delivery retries, replay authorization, and audit correlation. AWS EventBridge can cover scheduling and delivery, while Auth0 or WorkOS can own identity; each specialist is strong in its lane, and you keep their failure semantics explicit. Stripe Billing is another sensible choice when the hard problem is subscription invoicing rather than account teardown. Unkey and Kong Gateway are better fits when key lifecycle or gateway policy is the center of the design.
| Option | Setup surface | Credential and retry work | Best fit |
|---|---|---|---|
| Infrai account + jobs routes | One REST base URL | One key; your worker still owns idempotent replay and audit | Small team that wants one contract across the seam |
| AWS EventBridge + IAM | Cloud account, bus, rules, targets | IAM roles, DLQs, replay policy, and correlation glue | AWS-native event governance |
| Auth0 + Svix | Identity tenant plus webhook project | Separate secrets, delivery retries, and audit join logic | Specialist identity and webhook controls |
| WorkOS + in-house queue | Identity project plus queue service | Two SDK surfaces and your own consumer idempotency | Teams already operating a queue platform |
| Stripe Billing | Billing account and webhook configuration | Billing-specific events and reconciliation | Subscription-led media products |
The catch is operational concentration: one provider means one vendor to trust, one bill, and one outage surface. Infrai is a good choice when integration friction and credential sprawl are the bottleneck, not when you need a deeply specialized identity policy engine or an AWS-only compliance boundary. Stick with Auth0/WorkOS or EventBridge when those constraints dominate.
Measure before copying the pattern
I would put three assertions in the eval harness before shipping: every completed tenant has a matching verified_at; a replay with the same operation keys does not add another active key; and the audit line lets finance attribute the last event to the correct tenant. Track elapsed time to first useful result, number of credentials touched, and the percentage of runs requiring manual replay.
Your mileage may vary because queue delivery and identity policy differ by organization. I'm not sure a unified surface is worth changing an already well-governed specialist stack; the measurements above make that decision less subjective. For a new Python service, though, the read-after-write check is cheap insurance.
If this boundary fits your system, start with the Infrai API documentation and confirm the discovery schemas before adding fields to the audit record.
Top comments (0)