My Hermes agent runs on GLM 5.3 Flash hosted on Fireworks, and it’s more than capable of doing anything I throw at it. So naturally, adding another model sounds like another account to maintain and another place to look when a request stalls.
That kind of changed when I read more about Jev. It caught my attention because it takes on a smaller job. Before Hermes writes anything, it often has to choose which instructions to load or which tool to use. Those decisions happen repeatedly, even when the task itself is routine.
TypeSafe built Jev for that kind of judgment. You supply the available answers, and Jev returns a decision with probabilities your software can use.
I haven’t integrated Jev into Hermes yet, but I’ve been looking through TypeSafe’s documentation and the community integrations to work out where I’d try it. Skill selection looks like the strongest starting point, especially for an agent with a crowded catalog.
And I can already see a few places where Jev could help my Hermes workflow. I’ll walk through where I’d try it and what would make it worth another API bill.
Subscribe for practical agent experiments, including what earns a permanent place in the stack.
In this article:
What Jev does and where it fits
Four integration ideas for Hermes
A first experiment worth running
The limits I’d check before relying on it
What Jev Does
TypeSafe introduced Jev in early access on September 15. The company calls it a System One model, designed for quick judgments inside software.
You send it state, such as a user request or an error report, alongside questions with defined answer types. Its API has three primitives:
Choice selects from options you provide, returning a probability for each.
Score rates the input against ordered descriptions in your rubric.
Noul returns a number between zero and one representing the probability of “yes.”
Suppose Hermes receives: “Find the deployment instructions for this project.” A Choice question could select between a documentation-search tool and a terminal tool, with an explicit “none fits” option.
Jev would return the selected option. The main model would still have to prepare the search and interpret what came back.
Choice and Score also return a confidence value derived from how concentrated the probabilities are. That value tells your code how decisive the answer looks. A confidence of 0.9 shouldn’t be read as proof that 90% of equivalent decisions will be correct in your workflow. You need your own examples to establish that relationship. TypeSafe explains the distinction here.
The current model page lists Jev 1.13 at $0.042 per million input tokens , with output tokens free. The total request budget is 64k tokens, subject to a separate 32k limit for the state plus the longest question. Input is text only.
At that price, repeated classification becomes cheap enough to investigate. Whether it saves money overall depends on what it replaces and how often a wrong route creates extra work.
Where the Call Goes
Jev fits beside the model running Hermes. The main model still handles open-ended reasoning and writing. Jev answers a narrower question at a point where the workflow needs to branch.
For model routing, the proposed sequence is:
User request → Jev chooses a route → code checks the result
↓
selected model or existing fallback
Think of a support request that mentions a failed payment without using the word “billing.” A decision model can judge its meaning, while ordinary code sends it to the corresponding queue.
Exact conditions belong in code. If a spending limit is $10, compare the amount directly. Asking another model whether $12 exceeds the budget would be an expensive way to introduce uncertainty.
Hermes supports plugins and lifecycle hooks where these calls can run. The right connection depends on whether Jev should be available on demand or consulted automatically. That’s the same choice behind my MCP, CLI, or custom tool article.
Four Places I’d Try It
The Hermes catalog already includes several community Jev integrations. Catalog entries pin specific commits, but the review covers metadata and declared capabilities. Inclusion doesn’t amount to a full code audit or evidence that the plugin improves your agent.
Each of these ideas needs its own test.
Finding the Right Skill
An agent with a large skill catalog has to choose from descriptions that often resemble each other. “Create a presentation” and “edit a presentation” look close until the wrong instructions send the task sideways.
TypeSafe’s Hermes skill-suggestion cookbook tackles this with two requests. Jev first ranks the catalog, then reads more detail about the top three candidates. It can reject the shortlist when nothing fits.
The cookbook reports that wrong skill loads dropped from 16.8% to 7.3% across the covered requests in a 488-request evaluation. Loading a skill unnecessarily fell from 9.8% to 4.0% on requests with no matching skill.
Those are TypeSafe’s results, using Jev 1.12 with Claude Haiku 4.5 and a fixed Hermes roster. The evaluation includes generated requests, so it leaves plenty to check against everyday use.
The community jev-skill-router plugin implements this suggestion pattern. I’d compare its recommendation with Hermes’s existing choice before letting it influence a turn.
For a request to create a new slide deck, I’d want the authoring skill selected. Loading the editing skill would count as a visible miss, even if the agent eventually recovered.
Choosing a Model
A routing layer could send routine formatting to a cheaper model and reserve a stronger one for difficult work. Jev would select from routes the operator has already configured.
The descriptions need to reflect the task’s consequences. “Change one line in the production deployment config” is a short request with more at stake than rewriting a long document.
I’d give each route a precise description and include an uncertain outcome that keeps the existing model. Budget enforcement would stay in code.
A useful test would compare total cost per completed task. Saving on the first call accomplishes little if the cheaper model needs several repair attempts. I’d also inspect the tasks Jev sent down the cheap route incorrectly, because those failures reveal where the descriptions are too generous.
I use that same approach when testing models through a real Hermes workday. The whole task has to finish before the cheaper route earns its place.
Reviewing Tool Calls
Hermes already has an auxiliary-model approval path. The community jev-approvals plugin connects Jev to that part of the system.
A narrow question might ask whether a proposed terminal command changes files outside the approved project directory. The state would include the command and its resolved working directory, giving the judgment something concrete to inspect.
I’d evaluate recorded commands first, with the existing approval rules left in charge. A classification that overlooks an external write is an immediate failure, regardless of how confident the model sounds.
My approval-gate workflow still needs a human decision where the consequences warrant one. Jev’s score wouldn’t supply permission to send a client message.
This is also a place to retain exact checks. If code already knows that a path is outside the workspace, it should enforce that boundary directly.
Filtering Retrieved Context
An agent retrieving old project notes often gets more text than the next task needs. Jev could judge which passages deserve space in the main model’s context.
For example, a request about the current deployment might retrieve both an old setup note and its replacement. A useful filter would retain the current instructions and recognize that the earlier note was superseded, provided that evidence is present.
Dropping the current note would be harder to spot than loading the wrong skill. The agent might proceed confidently with the only version it received.
The community hermes-jev plugin includes context curation with shadow modes for observing proposed changes. I’d inspect those decisions before applying them, especially when a discarded passage contains a constraint.
There’s a data tradeoff here too. The hosted service receives the content being evaluated. TypeSafe says it doesn’t train on customer requests or responses, while its model documentation describes zero data retention for enterprise customers. Those are different commitments.
If you’re building your own agent setup, subscribe to All Agents Considered for practical ways to improve it without spending every weekend maintaining it.
A First Experiment
I’d start with skill selection on one Hermes profile, using a small set of real requests. During the first pass, Jev would only produce a log for comparison.
I’d collect around 50 requests and label the appropriate skill myself, including cases where no skill applies. That’s enough for an exploratory check, though it wouldn’t establish reliability across every kind of task.
Each record would contain the request and both selections. I’d add a short review note whenever they disagreed, so I could return to the reason behind the label.
Ambiguous cases deserve attention. A request to “improve these slides” may require clarification before either selector has a fair chance. Forcing a correct-answer label onto unclear input would make the comparison look more precise than the input warrants.
After refining the question descriptions, I’d evaluate a fresh set of requests. Otherwise, I’d be measuring how well I’d tuned Jev to the examples I’d already seen.
Only then would I let the suggestion reach Hermes. Even a correct suggestion can be ignored, so the second stage needs to measure which skill the agent loads and whether the task completes properly.
The fallback would preserve the current selection process whenever Jev fails or the answer is uncertain. I’d keep the questions and threshold settings together in one file, where changes are easy to review.
The result I care about is fewer wrong skill loads at an acceptable delay. Cost belongs beside that result, including any extra work caused by a bad suggestion.
Share this with someone considering Jev for their agent. A small comparison gives them somewhere concrete to start.
Using Other Agents
TypeSafe provides Python and JavaScript SDKs, so the same decision call can sit inside a custom agent application.
Its coding-agent skill gives tools such as Claude Code and Codex documentation for writing those integrations. Installing it supplies instructions. Runtime calls still need an integration and API credentials.
For an agent that supports external tools, an MCP wrapper is one possible way to expose Jev on demand. Automatically consulting it before every turn requires an extension point in that particular agent. The connection should follow the host’s supported interfaces.
The distinction is useful when asking a coding agent to build this. Specify the decision you want to test and where its answer will be consumed. “Add Jev to my agent” leaves most of the design undecided.
For Hermes, the community jev-typesafe plugin exposes explicit tools such as jev_route and jev_score. That offers a smaller entry point: ask the agent to evaluate one saved request, inspect the returned answer, and compare it with your own judgment before considering automatic calls.
For a custom agent, I’d put the equivalent call behind a small function that returns a suggested route. The caller would check that the route exists and remains allowed for the current task. Credentials would stay in the environment. Keeping execution outside that function makes it easier to replace Jev later without rewriting the workflow around it.
The Limits Worth Reading
TypeSafe’s Jev 1.13 limitations page is worth reading before copying an integration.
It describes a model that reads instructions word for word. Implied conditions can be missed, and complicated chains of reasoning make answers less dependable. Sending a large amount of irrelevant text also hurts performance.
Numeric tasks are another weak spot. Counting and date comparisons belong in code, even when Jev helps interpret the language around them.
It also needs adversarial testing. A document being evaluated may contain instructions intended to influence the judgment. Constraining the response format doesn’t settle whether the selected answer is right.
That distinction gets lost in the launch phrase “zero hallucinations.” Jev constrains the available output, so a Choice answer comes from the supplied options. It can still select the wrong one with confidence.
For my stack, the hosted dependency is another consideration. An unavailable decision service needs to leave the workflow with a usable fallback. I’d record the returned model version during evaluation too, because changing models complicates comparisons.
It would also need a place in my Hermes maintenance routine. If Jev stops responding, I want to know before I start blaming the main model.
Where I’d Start
Jev looks worth testing when a recurring judgment already causes trouble. Skill selection gives me a concrete place to measure that, with decisions I can inspect before they affect the work.
For an agent with a handful of tools and no selection problem, I’d leave the setup alone. The same goes for a workflow whose relevant data has to stay on the machine.
I haven’t established that Jev will improve my Hermes stack. The docs give me enough reason to run a small comparison, and enough detail to recognize a disappointing result.
If it reduces wrong skill loads, I’d let it suggest a skill during real work and see whether that improvement survives. If keeping the selector useful becomes another weekly maintenance job, I’d have my answer too.
Which decision would you want Jev to handle in your agent? Tell me the task, and where your current setup gets it wrong.
Top comments (0)