How to Build a Fast Decision Layer for AI Agents with TypeSafe Jev

CoderBuffer — Tue, 22 Sep 2026 09:46:45 +0000

Most AI agent architectures use the same large language model for every task: planning, writing, tool selection, relevance checks, risk scoring, and permission gates. That works for a prototype, but it creates an expensive and difficult-to-test control loop.

A large part of an agent's workload is not generation. It is decision-making over a closed set of outcomes.

TypeSafe Jev is designed for that narrower job. Instead of asking a generative model to produce prose and then parsing the answer, an application asks for one of three typed decision primitives:

Choice — select one option from a named set.
Score — place an item on an ordered scale.
Noul — estimate the probability that a statement is true.

This article shows how to use those primitives as a fast decision layer inside an agent, while keeping permissions, irreversible side effects, and rollback in ordinary application code.

Type safety constrains the output shape. It does not guarantee that the model selected the correct answer. Production systems still need calibration, fallbacks, and observability.

A layered agent architecture

A useful agent can be separated into four layers:

Deliberation: a capable generative model plans, explains, and writes.
Fast decisions: Jev routes, filters, scores, and gates.
Deterministic execution: application code owns permissions, side effects, and rollback.
Fallback: a human or a stronger model handles high-risk and low-confidence cases.

This boundary makes the model replaceable while keeping the decision protocol stable.

The fast layer is not the agent's brain. It is closer to the agent's reflex system.

Which tasks belong in a fast decision layer?

A task is a good fit when most of the following are true:

The possible outputs can be defined before the request.
The decision happens frequently.
A wrong answer can be detected or safely reversed.
The task requires semantic understanding but not a long explanation.

Common examples include tool routing, search-result relevance, quality scoring, moderation triage, permission requests, and human-review gates.

Open-ended writing, multi-file planning, and exploration with unknown outputs should remain in the generative layer.

Example: split a compound judgment into atomic questions

Suppose a coding agent needs to compact its context. Summarizing every old tool result can damage paths, stack locations, error codes, and command arguments. A safer alternative is to preserve user and assistant messages and make retention decisions only for paired tool calls and results.

Instead of one vague prompt such as “Can I remove this history?”, ask two atomic Noul questions:

const questions = {
  call_t3: {
    type: "noul",
    instructions: "Is knowing this tool call still useful?"
  },
  result_t3: {
    type: "noul",
    instructions: "Must the full result remain available?"
  }
};

Application code can combine the two probabilities into three explicit actions:

if (keepResult >= resultThreshold) {
  return "keep";
}

if (keepCall >= callThreshold) {
  return "drop_result";
}

return "drop_call";

The model estimates semantic value. Deterministic code decides what the system actually does.

That separation matters. It prevents a probabilistic component from silently owning destructive behavior.

Preserve structure before asking the model

Before sending anything to Jev, structure the agent history in code:

Pair each tool call with its result by ID.
Pin the first message and the most recent messages.
Build only the state needed for the current judgment.
Batch independent decisions when the state budget is limited.
Remove calls and results as a pair so the transcript never contains orphaned entries.

This also reduces the amount of context sent to the decision model. A fast decision layer should not receive the full agent transcript by default.

Use progressive degradation, not random deletion

When the decision state exceeds its token budget, degrade it in a predictable order:

Shorten long tool arguments.
Keep the head and tail of long text values.
Replace old messages with explicit omission markers.
Merge consecutive old calls only as a last resort.

Every degradation stage should emit metrics. If the system has to collapse old messages on 70% of requests, the state builder—not the model—may be the real bottleneck.

Calibrate confidence with your own data

Example thresholds such as 0.5 or 0.8 explain control flow; they are not production defaults.

A better rollout process is:

Run the Jev decision in shadow mode without changing behavior.
Store the question version, model version, probability, proposed action, and actual outcome.
Label a representative sample.
Measure observed accuracy by probability band.
Select thresholds based on error cost and reversibility.
Fall back to the original path on timeouts, malformed responses, or low confidence.

For read-only actions, automation may be acceptable after validation. For reversible actions, uncertainty should usually preserve the original state. For deletion, payments, access revocation, or other irreversible effects, Jev should never be the only authority.

Jev vs structured output, tool calling, and classifiers

These techniques overlap, but they solve different problems.

LLM + JSON Schema is useful for infrequent, complex judgments that require explanation. It still pays the cost and latency of a full generation path.

Tool calling is a good fit for actions in the main agent loop, but the choice is still made by a generative model.

Traditional classifiers are excellent for stable labels with enough training data. They can be cheap and local.

Jev fits frequent, closed, semantically rich, and recoverable decisions where you want typed outcomes and probabilities without a prose-generation step.

The right architecture may use all four.

The reusable pattern

The most important lesson is not a specific threshold or compaction algorithm. It is the interface between probabilistic judgment and deterministic control:

Generation expresses → judgment routes → code executes

Structure the object first. Split compound judgments into atomic questions. Combine probabilities in code. Keep side effects behind deterministic policy. Preserve a complete fallback.

That pattern applies far beyond context compaction: tool routing, content moderation, search reranking, permission gates, quality checks, and escalation decisions can all use the same boundary.

For the complete architecture diagram, rollout checklist, and a deeper context-compaction case study, read the full guide: Build an Agent Decision Layer with Jev.

DEV Community: CoderBuffer