Code Controls, Jev Decides, OpenAI Reasons: A Hybrid Agent Architecture That Doesn't Waste LLM Calls
When we build an AI agent today, the default architecture puts an LLM in the middle of everything. The user sends a request, the LLM interprets it, reasons about what to do, picks a tool, generates arguments, reads the result, reasons again, and finally responds.
That approach works. But while experimenting with agents, I started asking a different question: does every decision inside an agent really need a general purpose LLM?
Consider a simple request:
"I forgot my password."
The application already knows the possible workflows:
reset_passwordunlock_accountcheck_ordercancel_orderopenai_reasoninghuman_support
We're not asking AI to invent a new solution. We just need it to answer: which of these known capabilities best matches the request? That's a very different problem from asking an LLM to investigate an outage, explain an architecture, or compare options.
This distinction led me to a hybrid agent architecture using Jev and OpenAI. The basic idea:
CODE = CONTROL
JEV = DECIDE
OPENAI = REASON
TOOLS = ACT
Instead of asking one model to do everything, the agent uses the right kind of intelligence for each step.
What Is Jev?
Jev is TypeSafe AI's first public System One Model a model built specifically for decisions inside software, not for open ended generation.
A general-purpose LLM works like this:
Context → LLM → Generate tokens → Explanation / Code / JSON / Tool Call / Answer
Jev works differently:
Application State → Jev → Typed Probabilistic Decision
TypeSafe describes this as unstructured state in, typed probabilistic decisions out.
Suppose our application sends the user request plus the available capabilities. Jev might return something conceptually like:
reset_password 1.00
unlock_account 0.00
check_order 0.00
cancel_order 0.00
openai_reasoning 0.00
human_support 0.00
The application executes reset_password(). Jev doesn't write a paragraph explaining what to do — its job is the bounded semantic judgment: which known capability best fits this context?
Jev Is Not Just a Smaller LLM
It's easy to assume Jev is a cheaper model used for classification, but that misses the point.
A general-purpose LLM is optimized for flexible language generation it can explain, reason, summarize, write, generate code, compare alternatives, plan, and troubleshoot. Jev focuses on a narrower interface: decisions whose structure the application already defines. TypeSafe's workflow examples use three decision primitives Choice, Score, and Noul.
- Choice - "Which team? Billing / Shipping / Technical / Account"
- Score - "How severe is this issue? Low / Medium / High / Critical"
- Probabilistic yes/no -"Should this transaction be escalated? Yes → 0.91, No → 0.09"
The application defines the decision space. Jev is most useful when the system already knows the possible actions but still needs semantic understanding to pick the right one.
Jev vs. an LLM
Two requests make the difference clear.
Request 1: "I forgot my password." The possible workflows are already known this is a bounded decision.
Request 2: "Our application authenticates successfully through SSO, but users start getting errors when their OAuth access token expires. Analyze what could be happening and explain what we should investigate."
There's no predefined answer here. The system might need to think about refresh tokens, token rotation, scopes, audience, session expiration, IdP configuration, or token caching. That's open ended reasoning.
| Area | Jev | General-Purpose LLM |
|---|---|---|
| Primary role | Decision | Reasoning + generation |
| Answer space | Defined beforehand | Potentially open-ended |
| Output | Typed decisions | Generated content |
| Probabilities | Core part of interface | Possible, not primary |
| Long explanation | Not the purpose | Strong fit |
| Semantic routing | Strong fit | Possible |
| Tool selection from known set | Strong fit | Possible |
| Troubleshooting | Limited | Strong fit |
| Content creation | Not the purpose | Strong fit |
| Planning | Bounded | Open-ended |
The right question isn't Jev versus OpenAI, which is better it's which one fits the decision I'm making right now?
Why Build a Hybrid Agent?
An agent doesn't contain only one kind of problem. Inside the same interaction we usually have deterministic rules, semantic decisions, open ended reasoning, and real world actions. Forcing all four through the same model is wasteful. Instead, separate the responsibilities:
USER
│
▼
CONVERSATION STATE
│
▼
JEV
Decide / Route
│
┌─────────────┼──────────────┐
│ │ │
▼ ▼ ▼
Deterministic OpenAI Human
Workflow Reasoning Support
│ │
└──────┬──────┘
│
▼
TOOLS & SYSTEMS
│
▼
UPDATE STATE
│
▼
RESPONSE TO USER
Figure 1 Jev decides, OpenAI reasons, code orchestrates, and tools execute.
Let's Build It
The implementation is surprisingly small. Start with two API keys.
Step 1: Environment Configuration
TYPESAFE_API_KEY=your_typesafe_api_key
OPENAI_API_KEY=your_openai_api_key
OPENAI_MODEL=gpt-6-astra
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
TYPESAFE_API_KEY = os.getenv("TYPESAFE_API_KEY")
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-6-astra")
openai_client = OpenAI(api_key=OPENAI_API_KEY)
OpenAI's current Responses API uses client.responses.create(...) and exposes the generated text through response.output_text.
Step 2: Create a Capability Registry
Instead of a giant if/elif routing block, register capabilities so they can describe themselves:
TOOLS = {}
def register_tool(name, description):
def decorator(func):
TOOLS[name] = {
"description": description,
"function": func
}
return func
return decorator
@register_tool(
name="reset_password",
description="Use when the user forgot their password or wants to reset their password."
)
def reset_password(message, step=None):
return "Password reset workflow started."
@register_tool(
name="unlock_account",
description="Use when the user's account is locked or needs to be unlocked."
)
def unlock_account(message, step=None):
return "Account unlock workflow started."
We can also register the LLM itself as a capability:
@register_tool(
name="openai_reasoning",
description=(
"Use when the request requires explanation, analysis, comparison, "
"troubleshooting, summarization, writing, or open-ended reasoning."
)
)
def openai_reasoning(message, step=None):
...
This is the part I find most interesting: OpenAI becomes one capability available to the agent, rather than automatically being the whole agent.
Step 3: Dynamically Build Jev's Choices
The tool descriptions become Jev's decision space:
def build_jev_choices():
return {
name: config["description"]
for name, config in TOOLS.items()
}
Add a new capability — say refund_order — and it automatically becomes another candidate. No router rewrite needed.
Step 4: Let Jev Route the Request
from typesafe_sdk import Choice, TypeSafeClient
def route_with_jev(user_message):
choices = build_jev_choices()
state = {
"user_request": user_message,
"available_capabilities": list(choices.keys())
}
with TypeSafeClient() as client:
response = client.system_one(
state=state,
questions={
"route": Choice(
instructions=(
"Choose the single best capability for handling the "
"user's request. Use openai_reasoning when the task "
"requires explanation, analysis, troubleshooting, "
"comparison, writing, or open-ended reasoning."
),
criteria=choices
)
}
)
return response.choices["route"]
Jev's job is now narrow: given the request and the available capabilities, return the best one.
Example — password reset: the user says "I forgot my password," Jev sees the six capabilities, and selects reset_password with a confidence of 1.0. Job done.
But Routing Alone Doesn't Make an Agent
My first implementation just returned "Password reset workflow started." Technically correct, not very useful. A real agent needs to continue the conversation:
You: I forgot my password.
Agent: I can help with that. What email address is associated with your account?
You: sreeni@example.com
Agent: I've initiated the password-reset workflow. Please check your registered email.
That means the agent needs state.
Step 5: Add Conversation State
SESSION = {
"pending_tool": None,
"pending_step": None,
"data": {},
"history": []
}
After Jev chooses reset_password, the app stores pending_tool = "reset_password" and pending_step = "awaiting_email". When the user replies with an email address, we don't send that back to Jev the application already knows what it means, because it's answering the question the active workflow just asked.
This led me to a principle I keep coming back to: don't ask AI to decide something your application already knows.
Example — order cancellation follows the same shape: Jev selects cancel_order once, up front, and everything after that (asking for the order number, confirming, executing) is normal software. We don't need an LLM to understand "Yes."
Step 6: Add OpenAI as the Reasoning Capability
@register_tool(
name="openai_reasoning",
description=(
"Use when the user's request requires open-ended reasoning, "
"explanation, analysis, comparison, summarization, "
"troubleshooting, or generation."
)
)
def openai_reasoning(message, step=None):
response = openai_client.responses.create(
model=OPENAI_MODEL,
instructions=(
"You are the reasoning component inside a hybrid AI agent. "
"Provide a clear and concise answer."
),
input=message
)
return response.output_text
OpenAI is no longer called for every request. Jev decides, up front, whether this matches a known workflow or whether it actually needs open ended reasoning.
Example a request that actually needs an LLM: "Our application works with SSO initially, but starts failing after the access token expires. Explain what could be wrong and what we should investigate." Jev evaluates the six capabilities and selects openai_reasoning. Only now does OpenAI get involved, and it can discuss refresh-token expiration, token rotation, scope problems, audience, session expiration, IdP configuration, and caching behavior exactly the kind of work where an LLM earns its cost.
Two Architectures, Side by Side
A conventional LLM-centric agent:
USER → LLM understands → LLM reasons → LLM chooses tool → TOOL
→ LLM reads result → LLM reasons again → ANSWER
The hybrid architecture:
USER → JEV → Which capability?
├── Deterministic → CODE
├── Open-ended → OPENAI
└── Uncertain → HUMAN
→ TOOLS → RESULT
Neither is universally better. They optimize for different workloads.
Step 7: The Core Agent Loop
def run_agent(user_message):
# Continue an existing workflow
if SESSION["pending_tool"]:
return execute_tool(
SESSION["pending_tool"],
user_message,
SESSION["pending_step"]
)
# Otherwise ask Jev to route
decision = route_with_jev(user_message)
selected = decision.choice
confidence = decision.confidence
# Low confidence: ask for clarification
if confidence < 0.50:
return (
"I'm not completely sure what you'd like me to do. "
"Could you provide a little more detail?"
)
# Execute selected capability
return execute_tool(selected, user_message)
That's the whole architecture in one function: is a workflow already active? If so, continue it no LLM call needed. If not, ask Jev which capability fits, then route to code, OpenAI, or a human.
Where Confidence Fits
One thing I like about Jev's interface is that decisions come with probabilities and confidence as first-class outputs, not an afterthought.
If Jev returns **reset_password**: 0.97, the app can safely continue. But if it returns something closer to **reset_password**: 0.43, **unlock_account**: 0.39, **human_support**: 0.18, the harness shouldn't just pick the highest score blindly it can ask a clarifying question instead:
if confidence >= 0.85:
execute()
elif confidence >= 0.50:
ask_for_clarification()
else:
escalate_to_human()
The exact thresholds need to be tuned per use case rather than treated as universal.
Typed Doesn't Mean Semantically Correct
There's an important limitation here. If the allowed outputs are **reset_password**, **unlock_account**, and **human_support**, Jev will always return something structurally valid but a structurally valid decision can still be the wrong one. The model might select **reset_password** when **unlock_account** was correct. Both are valid outputs; only one matches reality.
Typed output does not mean guaranteed correctness. The application still needs evaluation, confidence policies, validation, guardrails, human escalation, observability, and testing. That doesn't go away just because the decision is typed.
Four Responsibilities, Not One "AI Problem"
The experiment got clearer once I stopped treating everything as an AI problem. There are really four distinct responsibilities:
CODE = CONTROL — use normal code when the rule is already known (if order.status == "SHIPPED": cancellation_allowed = False). There's no reason to ask a model to rediscover a deterministic business rule.
JEV = DECIDE — use Jev when the possible outcomes are known but semantic understanding is required: which workflow, which tool, which specialist agent, relevant or not, retry or stop, escalate or continue. This is bounded semantic judgment.
OPENAI = REASON — use a general-purpose LLM when the problem requires analysis, explanation, planning, generation, comparison, or synthesis for example, "analyze the last five production incidents and identify recurring failure patterns." That's not choosing from a menu; it requires reasoning across information.
TOOLS = ACT — tools perform the real-world action: Okta, Entra ID, ServiceNow, Jira, Salesforce, OpenSearch, Qdrant, databases, REST APIs, MCP servers, A2A agents. The model shouldn't just say "the password has been reset" unless the identity provider actually did it.
Above all four sits the agent harness, which orchestrates.
Where This Gets More Interesting
Password reset is intentionally a simple example. The architecture gets more interesting in real agent systems, where an agent is repeatedly asking bounded questions: which tool should I use, which MCP server should receive this request, is this retrieved document relevant, should I retry this failed operation, which specialist agent should handle this task, should this action require human approval, has enough evidence been gathered? Many of these don't need a full general-purpose reasoning model every time.
Jev + MCP. One obvious next step is dynamic MCP tool discovery — instead of manually registering capabilities, the system pulls them from an MCP server's list_tools() call, and Jev's decision space becomes dynamic:
MCP SERVER → list_tools() → Tool names + descriptions → JEV → Select best tool → call_tool()
Jev + multi-agent systems. The same idea works one level up — instead of selecting a tool, Jev selects a specialist agent:
ORCHESTRATOR
│
▼
JEV
│
Which specialist?
│
┌─────────────┼─────────────┐
▼ ▼ ▼
Finance Supply Engineering
Agent Chain Agent Agent
The choices are known. The intelligence is in picking the one that fits the current context.
So When Should I Use Jev?
Ask: can I clearly define the possible answers before making the model call? If yes, this is probably a good candidate for bounded semantic judgment — approve/review/reject, billing/shipping/technical, retry/stop/escalate, low/medium/high risk.
If you can't define the answer space because the problem requires exploration, explanation, synthesis, or creation, a general-purpose LLM is the better fit something like "analyze everything we know about this outage and identify the three most plausible root causes" is genuinely open ended.
The Bigger Lesson
For years, AI architecture discussions started with which LLM should we use? Agentic systems push us toward a better question: what kind of intelligence does this particular step require?
Sometimes the answer is code. Sometimes Jev. Sometimes OpenAI. Sometimes no model at all just call the tool. That produces a more modular architecture:
AGENT HARNESS
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
CODE JEV OPENAI
Control Decide Reason
│ │ │
└──────────────┼──────────────┘
│
▼
TOOLS
│
▼
ACT
I don't think the future of AI agents means finding one giant model and letting it control every decision. A more interesting architecture uses the right intelligence for the right decision: code when the rule is deterministic, Jev when the choices are known but semantic judgment is required, OpenAI (or another LLM) when genuine reasoning or generation is needed, and tools when something actually has to happen in the real world.
Code controls. Jev decides. OpenAI reasons. Tools act. The agent harness orchestrates everything.
That's the hybrid agent architecture I wanted to explore.
Agent Output
Thanks
Sreeni Ramadorai
Top comments (0)