DEV Community

FinnOakley52947
FinnOakley52947

Posted on

Node.js Feature Flags: REST API Implementation for Fintech Agent Telemetry

#observability #node #featureflags #fintech

Short answer: use a REST API to publish a revisioned flag snapshot, evaluate it locally in Node.js, and emit one decision event per agent step so latency and cost can be reconstructed later.

A feature flag is useful only if an incident investigator can answer one boring question: which decision did this agent make, and why? For a fintech agent loop, I would choose a small REST control plane plus an append-only decision event. The control plane changes policy; the event records the evaluated result, input facts, latency, and estimated cost. That split makes reconstruction possible without turning every request into a config fetch.

The trade-off is deliberate: a pinned snapshot can be stale, but a remote evaluation on every step can change behavior mid-run. Incident reconstruction needs a stable timeline more than it needs imaginary freshness.

Design choice Better for incident reconstruction Cost or risk
Fetch policy on every loop step Fresh changes Adds network latency and a new failure path
Pin a policy snapshot per run Stable replay Rollback takes effect on new runs, not old ones
Emit only final outcome Low event volume Hides the step that caused the spend or delay
Emit one event per evaluation Explains the timeline Requires a retention and redaction policy

Freshness is not free.

My recommendation is to pin a snapshot at the start of an agent run, evaluate locally, and emit one structured event per decision. Include a policy revision, not the policy document itself. This keeps the hot path predictable while giving operations a precise join key for logs, traces, and billing records.

What must an investigator reconstruct?

A flag is not a boolean in this system. It is a decision function over a subject, a context, and a policy revision. The minimum useful record is therefore larger than { enabled: true }:

type FlagDecision = {
  runId: string;
  stepId: string;
  flagKey: string;
  enabled: boolean;
  policyRevision: string;
  reason: "rule_match" | "default" | "invalid_context";
  evaluatedAt: string;
  latencyMs: number;
  estimatedInputTokens: number;
  estimatedOutputTokens: number;
};
Enter fullscreen mode Exit fullscreen mode

The identifiers matter. runId groups the agent loop, while stepId distinguishes retrieval, authorization, model invocation, and settlement checks inside that loop. policyRevision lets an investigator tell a policy change from a code deployment. reason catches the dangerous case where a missing account attribute silently selects the default.

That is ten fields for one event. It is enough to explain a decision without copying sensitive request content into the log stream.

I started by treating the flag service as the source of truth for every step. That looked clean. It also made a slow control-plane request look like model latency, and a transient timeout could change behavior halfway through one run. The correction was simple: resolve the snapshot once, carry it through the run, and record the resolution boundary.

How does a Node.js implementation keep feature flags behind a REST API?

The REST API needs only a narrow contract. A read returns the current revision and rules. An update accepts a new document with an explicit revision. The application should reject a stale update instead of silently overwriting a newer policy. The evaluator remains a local function, so it can be tested without HTTP.

type FlagRule = {
  flagKey: string;
  enabled: boolean;
  accountTier?: "standard" | "regulated";
};

type FlagSnapshot = {
  revision: string;
  rules: FlagRule[];
};

const snapshot = await fetch(
  "https://flags.internal/flags/snapshot",
  { headers: { accept: "application/json" } }
).then(async (response) => {
  if (!response.ok) throw new Error(`flag snapshot failed: ${response.status}`);
  return response.json() as Promise<FlagSnapshot>;
});

function evaluate(
  snapshot: FlagSnapshot,
  flagKey: string,
  accountTier: FlagRule["accountTier"]
): FlagDecision {
  const rule = snapshot.rules.find(
    (candidate) =>
      candidate.flagKey === flagKey &&
      (candidate.accountTier === undefined || candidate.accountTier === accountTier)
  );

  return {
    runId: currentRunId,
    stepId: currentStepId,
    flagKey,
    enabled: rule?.enabled ?? false,
    policyRevision: snapshot.revision,
    reason: rule ? "rule_match" : "default",
    evaluatedAt: new Date().toISOString(),
    latencyMs: 0,
    estimatedInputTokens: 0,
    estimatedOutputTokens: 0
  };
}
Enter fullscreen mode Exit fullscreen mode

The example leaves currentRunId and currentStepId to the surrounding request context. That is deliberate. A flag helper should receive explicit context in production rather than reaching into a global mutable object. The fetch timeout, retry budget, authentication, and response schema validation belong at the boundary too. They are operational policy, not evaluation logic.

A POST endpoint for recording the decision should be asynchronous from the agent's point of view when the reliability contract permits it. If the event is required for financial audit, use a durable local queue and make delivery state visible. Do not pretend that a fire-and-forget promise is durable. The limitation is important: asynchronous delivery can lose evidence during a process crash unless the queue has a durability guarantee.

Which signal makes a cost spike explainable?

Latency and cost need the same timeline. Record the flag decision before the model call, then attach the model measurement to the same runId and stepId. For token accounting, retain the provider-reported usage when available; estimates are useful for early detection but should be labeled as estimates. Never put prompts, account numbers, or payment data into a general-purpose event by default.

OpenTelemetry describes logs as timestamped records with attributes and context, which is a useful mental model for these decision events. Put runId, stepId, and policyRevision in structured attributes. Carry trace context where the logging pipeline supports it. Keep high-cardinality values intentional; an account identifier may be necessary for a restricted audit store but inappropriate for a broad dashboard.

The practical test is reconstruction, not dashboard prettiness. Given a run ID, I should be able to produce this sequence: snapshot revision, flag decision, model call, tool calls, outcome, and recorded usage. If one of those links is missing, the system may still be observable, but it is not explainable under pressure.

Replay wins.

When is a different design the better choice?

A per-request evaluation can be the right choice when policy changes must affect an already-running workflow immediately, such as an emergency authorization block. In that case, the event must record the exact policy revision for every step, and the workflow must tolerate a decision changing between steps. The freshness benefit is real. So is the complexity. It is a poor fit for replay-sensitive batches because two steps can observe different policy revisions.

A fully local snapshot can be better for a disconnected worker or a tightly bounded batch. Its weakness is stale policy. Give it an expiry and a clearly recorded fallback reason. A remote evaluator can centralize complicated targeting rules, but it puts availability and latency directly into the agent loop. Pick it only when central enforcement outweighs replay simplicity.

The decision rule is narrow: optimize for the failure you need to reconstruct. For a fintech agent measuring latency and cost, stable run-level policy plus per-step decision events usually gives the cleanest evidence. Keep the REST surface small, reject stale writes, and make defaults observable.

Sources

Top comments (0)