Short answer: check the flag at the start of each agent decision, then emit one structured event containing the evaluated key, subject, result, and correlation identifiers. For a property-management agent, that event is more useful than a later log saying an escalation was enabled. It lets you reconstruct which behavior the agent actually saw while handling a resident message, maintenance request, or lease question.
The least complex design is a small Node.js adapter around your existing flag evaluator. Keep evaluation separate from recording. The adapter should return the boolean the agent needs and write an audit event with the same trace and request context. Do not put the flag state only in a dashboard screenshot; a dashboard can show an aggregate while an incident needs one decision from one agent run.
The event is the evidence.
What does an enabled flag prove during an incident?
It proves only what was evaluated for that subject at that moment. A flag named maintenance_triage_v2 being enabled globally does not prove that a particular building, property manager, or conversation received the new path. Targeting rules, a stale local cache, a default value, and a failed evaluation can all change the decision.
That distinction matters in an agent loop. A single resident request may cause several model calls and several tool decisions. If the first turn used the old maintenance workflow and a retry used the new one, a final response log cannot explain the transition. Record the decision at the boundary where the application chooses behavior. For example, an incident reviewer should be able to follow request r-88: the evaluator returned false for maintenance_triage_v2 on property p-104, the agent used the existing routing branch, a retry evaluated the same key again, and the subsequent tool call was made under the same trace. Without both evaluations, the reviewer cannot tell whether the behavior changed, the request retried, or the log merely arrived out of order. That is why I would keep one event per evaluation even when the values repeat; deduplication saves storage while removing the exact evidence the incident question needs.
Here is a deliberately vendor-neutral interface. The values in the example event are illustrative test data, not a benchmark or a claim about a production system. The adapter uses fetch, so it runs on current Node.js releases that provide the standard Fetch API; replace the evaluator endpoint with the flag service already approved for your deployment.
type FlagContext = {
subject: string;
propertyId: string;
environment: string;
traceId: string;
requestId: string;
};
type FlagDecision = {
key: string;
enabled: boolean;
reason?: string;
};
type FlagEvaluator = {
evaluate(key: string, context: FlagContext): Promise<FlagDecision>;
};
type AuditEvent = {
type: "feature_flag.evaluated";
timestamp: string;
flag: string;
enabled: boolean;
reason: string;
subject: string;
property_id: string;
environment: string;
trace_id: string;
request_id: string;
};
function createAuditedEvaluator(
evaluator: FlagEvaluator,
emit: (event: AuditEvent) => Promise<void>,
): FlagEvaluator {
return {
async evaluate(key, context) {
const decision = await evaluator.evaluate(key, context);
const event: AuditEvent = {
type: "feature_flag.evaluated",
timestamp: new Date().toISOString(),
flag: decision.key,
enabled: decision.enabled,
reason: decision.reason ?? "unspecified",
subject: context.subject,
property_id: context.propertyId,
environment: context.environment,
trace_id: context.traceId,
request_id: context.requestId,
};
await emit(event);
return decision;
},
};
}
async function emitJsonLog(event: AuditEvent): Promise<void> {
process.stdout.write(`${JSON.stringify(event)}\n`);
}
The important property is not the endpoint shape. It is the event contract. enabled is the evaluated result, while flag identifies the decision and subject identifies whose context was evaluated. trace_id and request_id connect the event to the agent loop and the inbound API request. The property identifier keeps the incident query useful for a building or portfolio without forcing an operator to infer it from free text.
Emit after evaluation succeeds. If evaluation fails, choose a documented fail-open or fail-closed policy for that flag and emit a separate failure event with the policy outcome. Do not silently convert an evaluator exception into false; that makes an outage look like an intentional rollout decision.
How should the API preserve enough context without leaking resident data?
Start with identifiers, not prompts. A useful event can carry a stable subject reference, property ID, environment, decision reason, and correlation IDs without copying the resident's message, lease details, phone number, or model output. Those payloads have a different retention and access problem.
The event should also distinguish the evaluation result from the configuration state. If the evaluator returns a reason such as target_match, default, or error, preserve it as a bounded value. Avoid accepting arbitrary reason strings from a remote service into a high-cardinality metric label. Put the reason in the event body and keep metric dimensions small.
For the same reason, do not make request_id or property_id a metric label by default. They belong in logs or traces for lookup. Metrics should answer questions such as “how often did this flag evaluate false in production?” Incident reconstruction needs the individual event afterward.
OpenTelemetry describes logs as a signal that can carry a timestamp, severity, body, attributes, and context, and its log model supports correlation with traces. That makes a structured flag-decision event a better fit for a log record than a prose line such as new workflow on. The event can be exported through the logging path already used by the application, while the exact collector and storage choice remains an operational decision.
Which failure modes should the test cover?
Test the decision boundary as a contract. The useful cases are not limited to true and false: a targeted property, an unmatched property, an evaluator timeout, a malformed response, a repeated evaluation inside one trace, and a retry after a transient failure all produce different reconstruction questions.
One test should assert that the agent receives the evaluator result unchanged. A second should assert that the emitted event contains the same flag key, subject, property, trace, and request identifiers. A third should assert your explicit failure policy. These tests catch a common instrumentation mistake: recording a default value while the application actually used a cached or targeted result.
Use an in-memory emitter in unit tests and a local log collector or fixture in integration tests. The test data can use two properties and two flag outcomes; the point is to verify correlation and ordering, not to manufacture a performance claim.
If the agent evaluates the same key twice, preserve both events or add an attempt field. Collapsing them at emission time destroys evidence about a mid-request change. If volume becomes a concern, make sampling a storage decision after retaining the audit events needed for incident reconstruction. Sampling the only record of a safety-relevant routing decision is a hard trade-off, not an invisible optimization.
What should an operator query after a rollout?
Reconstruction starts with the inbound request or trace ID. From there, inspect flag events in timestamp order, then compare the selected path with tool calls, model calls, and the final response. For property management, the practical question might be: did the maintenance-triage path evaluate true for property p-104 during request r-88, and did the agent subsequently call the maintenance system?
Keep the query workflow boring. Use a stable event name, ISO 8601 timestamps, bounded outcome values, and one documented location for correlation fields. Add a schema version when the event contract changes. A seven-field event is easier to migrate than a collection of free-form messages, but the exact field count should follow the fields your incident process actually uses.
The operational checklist is short. Define the flag's failure policy; emit the evaluated result rather than a global setting; attach trace and request context; exclude resident content from the audit body; test targeted, default, retry, and error paths; and verify that an operator can follow one property request from API entry to agent decision. Rehearse the query before a rollout. If the answer requires guessing which cache or flag rule applied, the event is missing a decision fact.
Further reading
- OpenTelemetry, “Logs signal concepts”: https://opentelemetry.io/docs/concepts/signals/logs/
- OpenTelemetry, “Logs data model”: https://opentelemetry.io/docs/specs/otel/logs/data-model/
- W3C Trace Context: https://www.w3.org/TR/trace-context/
- Node.js documentation, Fetch API: https://nodejs.org/api/globals.html#fetch
Top comments (0)