Run the credential inventory as an unattended, read-only control, then archive a dated and integrity-protected review document outside the event ingestion path. The deciding constraint is the blast radius of one job credential: a reporting task must never gain permission to read game events, rotate production keys, or rewrite old evidence.
TL;DR: split the workflow into three boundaries: inventory read, evidence creation, and archive write. Record key metadata, never secret values. Fail closed on incomplete input, sign the exact bytes you store, and emit a tiny operational result that can wake a human without leaking the inventory into logs.
The mental model is a crisp before and after. Before, a scheduler wakes up with a broad platform credential, queries everything, and overwrites latest.json. An outage can leave partial evidence, while compromise of that credential can reach the live ingestion system. After, a narrowly authorized reader produces one immutable, dated document; a separate archive capability can create a new object but cannot replace an old one. The live event path stays out of reach.
How should a scheduled access review job isolate its key inventory?
Picture the system in words. Game clients send match and purchase events to an ingestion tier. That tier uses runtime credentials for queues, storage, and internal APIs. Beside it, not inside it, a scheduled review job reads a metadata-only inventory. The job converts that snapshot into review evidence and writes a new archive object. Monitoring receives only status, counts, duration, and a document identifier.
This separation matters most during an outage. The ingestion tier may be overloaded or unreachable, but the compliance job should not make recovery harder. It should have bounded retries and no authority to change runtime credentials. If its inventory source is unavailable, it records a failed run in operational telemetry and creates no review artifact. A missing document is honest. A partial document that looks complete is dangerous.
Stop there.
The three authorization boundaries are concrete:
- The inventory reader can retrieve credential metadata for named services and environments. It cannot retrieve secret material or mutate credentials.
- The evidence process can transform data in memory and sign the final bytes. Its signing capability is distinct from game-event credentials.
- The archive writer can create a date-stamped object under one prefix. It cannot overwrite prior objects or read the event stream.
There is a trade-off. More boundaries mean more policies and more failure signals to maintain. I would still choose them here because the primary risk is credential reach, not scheduler convenience. One broad token makes the happy path shorter and the incident scope much larger.
That is the decision rule.
Define evidence that can survive scrutiny
An inventory row should answer who owns a key, what it can reach, and whether a reviewer acted. It should not contain the key. OWASP's secrets-management guidance recommends tracking metadata such as ownership, purpose, creation, rotation, and expiration while protecting secret values throughout their lifecycle.
Use UTC timestamps with explicit offsets. Give every source record a stable identifier that is not the credential itself. Represent scope as data so a reviewer can spot a cross-environment credential without opening policy files.
Here is the document contract used by the example:
type KeyRecord = {
keyId: string;
service: "match" | "store" | "telemetry";
environment: "production" | "staging";
owner: string;
scopes: string[];
createdAt: string;
expiresAt: string | null;
lastUsedAt: string | null;
status: "active" | "disabled";
};
type ReviewDecision = {
keyId: string;
decision: "retain" | "rotate" | "disable";
reason: string;
};
type ReviewDocument = {
schemaVersion: 1;
reviewId: string;
generatedAt: string;
sourceSnapshotAt: string;
sourceComplete: boolean;
records: KeyRecord[];
decisions: ReviewDecision[];
};
sourceComplete is deliberately blunt. The producer refuses to archive unless it is true, but retaining the field in the signed schema makes the completeness assertion reviewable. The decision list may be empty when the job only prepares evidence for a human reviewer. Do not silently label absence of a decision as approval.
Scope is the fast blast-radius test. A production telemetry key with events:write is understandable. A single key spanning production match administration, purchases, and telemetry is a review finding because one disclosure crosses several trust zones. The document makes that visible without exposing any secret.
Build the unattended producer
The copyable example below uses only built-in Node.js modules. It reads a metadata snapshot from a file mounted by an inventory adapter, validates every record, sorts unstable arrays, creates canonical JSON bytes, signs those bytes with HMAC-SHA-256, and uses exclusive file creation so a previous archive cannot be replaced.
The signing key arrives through the process environment for brevity. In a deployed system, inject it through the platform's secret-delivery mechanism and grant it only to this workload. The input file must contain metadata only.
import { createHash, createHmac, randomUUID } from "node:crypto";
import { mkdir, open, readFile } from "node:fs/promises";
import { dirname, join } from "node:path";
type KeyRecord = {
keyId: string;
service: "match" | "store" | "telemetry";
environment: "production" | "staging";
owner: string;
scopes: string[];
createdAt: string;
expiresAt: string | null;
lastUsedAt: string | null;
status: "active" | "disabled";
};
type Snapshot = {
capturedAt: string;
complete: boolean;
records: KeyRecord[];
};
const isoDate = (value: string): string => {
const parsed = new Date(value);
if (!Number.isFinite(parsed.valueOf())) throw new Error(`Invalid timestamp: ${value}`);
return parsed.toISOString();
};
const assertRecord = (value: unknown): asserts value is KeyRecord => {
if (!value || typeof value !== "object") throw new Error("Record must be an object");
const row = value as Partial<KeyRecord>;
if (!row.keyId || !row.owner || !row.service || !row.environment) {
throw new Error("Record is missing identity or ownership metadata");
}
if (!Array.isArray(row.scopes) || !row.createdAt || !row.status) {
throw new Error(`Record ${row.keyId} is incomplete`);
}
isoDate(row.createdAt);
if (row.expiresAt) isoDate(row.expiresAt);
if (row.lastUsedAt) isoDate(row.lastUsedAt);
};
const stableRecords = (records: KeyRecord[]): KeyRecord[] =>
records
.map((record) => ({ ...record, scopes: [...record.scopes].sort() }))
.sort((a, b) => a.keyId.localeCompare(b.keyId));
async function run(): Promise<void> {
const inputPath = process.env.INVENTORY_PATH;
const archiveRoot = process.env.ARCHIVE_ROOT;
const signingKey = process.env.EVIDENCE_SIGNING_KEY;
if (!inputPath || !archiveRoot || !signingKey) throw new Error("Missing job configuration");
const startedAt = new Date();
const snapshot = JSON.parse(await readFile(inputPath, "utf8")) as Snapshot;
if (!snapshot.complete) throw new Error("Inventory source reported an incomplete snapshot");
isoDate(snapshot.capturedAt);
snapshot.records.forEach(assertRecord);
const generatedAt = new Date().toISOString();
const reviewId = randomUUID();
const document = {
schemaVersion: 1 as const,
reviewId,
generatedAt,
sourceSnapshotAt: isoDate(snapshot.capturedAt),
sourceComplete: true,
records: stableRecords(snapshot.records),
decisions: [],
};
const payload = `${JSON.stringify(document, null, 2)}\n`;
const signature = createHmac("sha256", signingKey).update(payload).digest("hex");
const digest = createHash("sha256").update(payload).digest("hex");
const day = generatedAt.slice(0, 10);
const outputPath = join(archiveRoot, day, `${reviewId}.json`);
await mkdir(dirname(outputPath), { recursive: true });
const handle = await open(outputPath, "wx", 0o600);
try {
await handle.writeFile(JSON.stringify({ document, integrity: { digest, signature } }, null, 2));
await handle.writeFile("\n");
} finally {
await handle.close();
}
const durationMs = Date.now() - startedAt.valueOf();
process.stdout.write(JSON.stringify({ status: "ok", reviewId, recordCount: document.records.length, durationMs }) + "\n");
}
run().catch((error: unknown) => {
const message = error instanceof Error ? error.message : "Unknown failure";
process.stderr.write(JSON.stringify({ status: "failed", error: message }) + "\n");
process.exitCode = 1;
});
The wx flag is a local filesystem guard, not a universal retention guarantee. A production archive should enforce create-only or retention behavior at its storage boundary, independently of application code. Keep the dated object as the evidence record. If operators need a convenient “latest” pointer, treat that pointer as mutable navigation, never as evidence.
The HMAC-SHA-256 call returns a 32-byte digest, rendered here as 64 hexadecimal characters. Store the algorithm identifier beside the value, as the example does implicitly through the field contract, so verification doesn't depend on institutional memory. The explicit trade-off is portability versus stronger key isolation: HMAC is easy to run in a small unattended job, but every verifier that holds the same key could also create a valid signature. If independent verifiers must never mint evidence, replace the shared-key signature with an approved asymmetric signing mechanism while preserving the exact-byte rule.
There is another small but important choice here: the job logs recordCount, not records. Logs often travel farther than the archive and are read by more people. Shipping owners, scopes, or key identifiers into general logging expands the disclosure surface for little alerting value.
Make failure observable without widening access
The job needs a service-level signal, even though it is a batch control. Track completion state, run duration, inventory age, record count, and the age of the newest successful archive. Alert on absence as well as explicit failure; a dead scheduler emits no error from the task itself. Keep dimensions bounded. environment and result are useful metric labels. reviewId, keyId, and owner belong in the protected document or a structured log with controlled access, not in metric labels. This keeps the monitoring channel useful without copying the inventory into another system. A retry must generate a new review ID. It may reuse the same complete source snapshot, but it must not overwrite the first attempt. That gives investigators a visible sequence. To avoid a retry storm during a broader platform outage, cap attempts, add randomized delay in the scheduler, and alert after the final failure. The job is compliance work; it shouldn't compete aggressively with match recovery.
Silence is a signal.
Test the unhappy path. Feed the producer malformed timestamps, duplicate identifiers, an incomplete snapshot, a missing signing key, and an existing output path. The sample already rejects most of these, but duplicate IDs deserve an explicit check in the inventory adapter because sorting alone does not resolve identity ambiguity.
What if the inventory service is down during the review window?
Do not substitute a partial live query and call it complete. Either consume the most recent authenticated snapshot under a documented freshness limit or fail the run. Which option is acceptable depends on the control owner's evidence policy, so encode the chosen maximum snapshot age in configuration and include the actual sourceSnapshotAt in every document.
For example, a team might set a 24-hour maximum during an outage and reject anything older. That number is a policy choice, not a universal security threshold; writing it down makes the stale-evidence trade-off testable.
The outage path should be rehearsed. Disable the inventory dependency in a staging exercise, confirm that no archive document appears, and verify that the missing-success alert fires. Then provide a complete prior snapshot and confirm that policy either accepts or rejects its age. This is where a dated archive earns its keep: the reviewer can distinguish collection time from report-generation time.
Never let the reporting job “fix” access while dependencies are unstable. Rotation and disabling are separate, approved workflows with different permissions and rollback concerns. The evidence can recommend rotate or disable; it must not quietly perform either action.
Can the archive prove who approved each key?
Not by itself. A machine-produced inventory proves what the collector observed and, with signature verification plus protected storage, helps detect later modification. Human approval requires an additional review event tied to the document digest, reviewer identity, decision, and time.
Append that decision as a new record rather than editing the inventory artifact. The pattern preserves two different facts: what the scheduled job captured, and what a reviewer later decided. Requiring the reviewer to sign or strongly authenticate the decision is a policy choice, but the linkage should always use the stored document digest rather than a mutable filename.
This also clarifies team ownership. Platform engineering owns complete metadata collection and the archive control. Security or compliance defines acceptable age, review cadence, and evidence retention. Service owners explain unusual scopes and act on decisions. The split introduces handoffs, yet it prevents the unattended collector from becoming an all-powerful credential administrator.
Three boundaries are enough to make the decision rule memorable: read metadata narrowly, create deterministic evidence, and write once to a separate archive. During a gaming outage, that design keeps compliance reporting observable while leaving live event recovery credentials alone. The report remains useful because it records completeness, time, scope, ownership, and integrity without ever collecting the secrets it reviews.
Top comments (0)