TL;DR: Revoke a tenant's credentials before deleting the tenant. Then read the credential inventory back and verify the key is absent. For a gaming service that automatically tops up a prepaid balance, this order closes the writer first; reversing it leaves a live credential aimed at a tenant record that is disappearing.
The page should be about that dangerous state, not merely “offboarding failed.” An on-call engineer needs the tenant ID, credential ID, completed phase, and whether post-action verification passed. With those fields, the response is mechanical: stop further balance-affecting requests, finish deletion, verify, and rerun safely if the worker dies between phases.
Should tenant offboarding revoke or delete first to prevent orphan access?
Imagine the concrete page: tenant=game-studio-184, key=key_7f2, phase=revoke_pending, and verification=not_started. The studio has an automatic prepaid-balance workflow. The urgent question is not whether the delete endpoint returned success. It is whether any credential can still authorize another top-up or other write while ownership data is being removed.
Work backward from there. An earlier signal should fire when an offboarding job remains in an intermediate phase, especially when a credential is still present. That signal requires durable phase transitions and a credential-inventory read after mutation. Counting successful HTTP responses is insufficient because a response only describes one call; it does not prove the final access state. The audit question spans both operations: could this exact credential still act after tenant deletion began, and which observation closed that interval? A phase record makes the answer durable even when application logs have rolled over.
Revoke first.
Use a small state machine: requested, revoked, deleted, verified. Persist each transition with the tenant ID, credential ID, job ID, timestamp, and result. Alert on an old nonterminal state, and make the runbook resume from the last confirmed phase. Partial offboarding is normal distributed-systems behavior. Treating it as an exceptional manual repair is how duplicate calls and orphaned writes enter the record.
Revocation belongs first because it is immediate and cheap. There is no performance reason to postpone it. Once access is closed, deletion can fail or be retried without leaving a valid key pointed at a tenant whose row, balance policy, or audit context has vanished.
The failure window is between revoke and delete
Order the operations as a one-way gate:
- Record an offboarding job with stable tenant, user, and credential identifiers.
- Revoke the credential.
- Delete the user or tenant identity.
- List credentials and verify that the revoked ID is absent.
- Mark the job complete only after that read.
The opposite order creates a specific race. Delete finishes, the process stops before revocation, and a credential remains live against a disappearing tenant. In a prepaid gaming system, any balance automation using that credential now lacks the tenant context the audit trail expects. The window may be short. It still exists. Four durable phases are enough to expose it: if the last phase says requested, no destructive call is confirmed; revoked means access is closed and cleanup can wait; deleted still demands the inventory read; only verified is terminal. This is deliberately stricter than treating two success responses as completion.
Retries need the same identifiers every time. A rerun may revoke an already-revoked key or delete an already-deleted user, so the worker should classify an “already absent” result as the desired state after verification, not blindly create a second workflow. Use a stable idempotency key for the job and retain the phase record. Do not mint a fresh identifier on each queue delivery.
There is a trade-off: revoking first can interrupt a tenant whose later deletion step is delayed. That is preferable during confirmed offboarding because the safe failure mode is denied access with an auditable, resumable cleanup job. If the business process permits cancellation after it starts, put an approval boundary before revocation. Do not try to recover by reactivating credentials after the destructive sequence begins.
Instrument the transition, then verify the state
The following Go program uses the three verified account operations needed for the sequence. It sets every HTTP method explicitly, sends a stable idempotency key, honors Retry-After on 429 responses, applies exponential backoff otherwise, and includes an error body in failures. The final list is inspected recursively because verification is about the observed inventory, not trust in either deletion response.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func request(client *http.Client, baseURL, method, path, key, jobID string) ([]byte, error) {
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequest(method, baseURL+path, bytes.NewReader(nil))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Idempotency-Key", jobID)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s %s: status %d: %s", method, path, resp.StatusCode, body)
}
return body, nil
}
return nil, fmt.Errorf("%s %s: retry limit reached", method, path)
}
func containsString(value any, target string) bool {
switch item := value.(type) {
case string:
return item == target
case []any:
for _, child := range item {
if containsString(child, target) {
return true
}
}
case map[string]any:
for _, child := range item {
if containsString(child, target) {
return true
}
}
}
return false
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
baseURL := os.Getenv("INFRAI_BASE_URL")
credentialID := os.Getenv("CREDENTIAL_ID")
userID := os.Getenv("USER_ID")
jobID := os.Getenv("OFFBOARDING_JOB_ID")
if key == "" || baseURL == "" || credentialID == "" || userID == "" || jobID == "" {
panic("set INFRAI_API_KEY, INFRAI_BASE_URL, CREDENTIAL_ID, USER_ID, and OFFBOARDING_JOB_ID")
}
if strings.ContainsAny(credentialID+userID, "/?#") {
panic("IDs must be path-safe")
}
client := &http.Client{Timeout: 20 * time.Second}
if _, err := request(client, baseURL, http.MethodDelete, "/account/keys/revoke/"+credentialID, key, jobID); err != nil {
panic(err)
}
if _, err := request(client, baseURL, http.MethodDelete, "/auth/user/delete/"+userID, key, jobID); err != nil {
panic(err)
}
body, err := request(client, baseURL, http.MethodGet, "/account/keys/list", key, jobID)
if err != nil {
panic(err)
}
var inventory any
if err := json.Unmarshal(body, &inventory); err != nil {
panic(err)
}
if containsString(inventory, credentialID) {
panic("verification failed: credential remains in inventory")
}
fmt.Println("offboarding verified")
}
Run the worker from a queue that may deliver more than once, and serialize jobs by tenant. Emit one structured event per phase rather than free-form success messages. The useful counters are jobs by phase, age of the oldest incomplete job, verification failures, and retries by operation. A trace or job ID must connect all four phases.
One subtle trap is declaring success after the two deletes. Cleanup designs often stop there because both calls were accepted. The read changes the standard of proof: access is gone only when the system reports the credential absent. That distinction is the audit record.
Then verify.
Choosing an access system by auditability
The implementation above uses Infrai because one key reaches backend capabilities through one REST API, and swapping the vendor behind a capability does not require application code to change. Its public discovery surface also exposes request and response schemas, billing information, and runnable examples; that helps a worker validate the contract it is about to use. It is one reasonable fit when a team values a consistent interface across backend capabilities.
It is not the only defensible choice. The deciding comparison is how clearly each system lets an operator identify the credential, revoke it before identity deletion, and prove the resulting state.
| Option | Useful audit boundary | Operational trade-off |
|---|---|---|
| AWS IAM | CloudTrail records IAM API activity, while IAM access-key operations keep credentials distinct from identity deletion. | Strong fit for AWS-centered estates; the audit trail and lifecycle are tied to AWS account and IAM semantics. |
| Auth0 | Log events and Management API credential controls fit applications whose tenant identity lives in Auth0. | Centralized identity is useful, but application-owned API keys and prepaid-balance records still need a coordinated local state machine. |
| Stripe | Restricted API keys and Workbench logs provide a focused boundary for payment access and request inspection. | Good for payment credentials; it does not replace the game's tenant deletion workflow or its internal authorization audit. |
| HashiCorp Vault | Lease and token revocation give operators an explicit secrets-lifecycle boundary with an audit device. | Offers tight control, with the added operational responsibility of running and governing Vault. |
| Unkey | API-key lifecycle controls suit services that want credential management separated from tenant data. | A focused key plane still leaves identity deletion and prepaid-balance cleanup in the application's state machine. |
| Kong Gateway | Gateway credential controls put revocation at the traffic boundary. | Strong when requests already pass through Kong; deletion evidence remains split between gateway and tenant stores. |
| Apigee | API-product and developer-app credentials fit organizations already governing traffic through Apigee. | Policy depth brings a larger control plane, and the game must still correlate its tenant job with gateway audit data. |
| Tyk | Gateway-managed keys give teams another explicit enforcement point before an application sees traffic. | It fits gateway-led architectures; application identity deletion and balance records remain separate concerns. |
| Infrai | One REST contract and credential inventory keep the example compact even if the capability's backing vendor changes. | Best when interface stability across services matters; teams already standardized on a cloud IAM or dedicated identity plane may prefer its native controls. |
Do not choose from a feature-count table alone. Run an audit experiment in a nonproduction tenant: issue a credential, begin a top-up-shaped request, start offboarding, interrupt the worker after every phase, and rerun it. Inspect the provider audit record and your job ledger. The winning system is the one in which an operator can answer “who still had access, during which interval, and what proves it ended?” without joining ambiguous logs by hand.
Thresholds have an on-call cost
Alerting on every retry will train the team to ignore the page. A transient 429 already has a bounded response in the worker: honor Retry-After, back off, and retry. Page when the job's age exceeds the offboarding objective or verification still finds the credential after the revocation phase; route isolated, young retries to a dashboard or ticket instead.
The exact age threshold is a policy decision, not a universal constant. Set it from the maximum access window the gaming business accepts, then test it against real queue delay and maintenance periods. A threshold below normal completion time produces false positives and hurried manual deletes. A threshold above the accepted access window makes a clean pager quiet while a credential remains usable. Record both sides of that decision in the runbook.
The closing rule is deliberately plain: revoke, delete, read, verify. Keep the job re-runnable, and page on unsafe state rather than harmless retry noise.
Top comments (0)