A production agent makes the same three kinds of calls all day: pick one thing from a list, put a value on a scale, answer yes or no. Most teams send all three to a model that writes paragraphs.
JEV is built for exactly those three. You declare the legal answers up front, and it returns a typed answer with a probability. Below are 18 places to test it. For most of them we point to a project in the Awesome JEV catalogue that already does something close, so you can read real code before you build.
The compact version is:
fully deterministic → code fuzzy input, bounded answer space → JEV open-ended writing or reasoning → LLM
Extracting an order number from a known format belongs in a parser or regex. Routing a support ticket to billing, technical, or sales is a plausible JEV task. Writing the actual response still belongs to an LLM or a person.
The 18 patterns below are a selection map. They show where a decision model is worth testing, where the model should stop, and what must remain in ordinary code or human review.
Three primitives
JEV returns three core answer types rather than open prose.
- noul: How likely is a proposition to be true? Returns a probability from 0 to 1.
- choice: Which item best fits from a declared set? Returns the selected item, the full distribution, and confidence.
- score: Where does this state fall on an ordered scale? Returns a continuous score, per-level probabilities, and confidence.
The small interface forces useful product decisions. You have to define which answers are legal, what probability is sufficient for automation, when the system must stop, and when a person takes over.
Agents and workflows
1. Agent completion checks
An agent saying “done” does not prove that the result meets the acceptance criteria. Put the goal, acceptance criteria, latest artifact, and execution evidence into the state, then ask separate noul questions for each criterion.
JEV supplies signals. Workflow code still decides whether to finish, retry, request more evidence, or escalate.
In the catalogue:
— judges goal completion or stalled progress inside a browser loop.
2. Tool and function selection
When an agent has a fixed tool set, a choice over real tool names is easier to validate than a model inventing a function name in prose.
Add a separate noul question for whether any tool should be called at all. Selection is not authorization. Permissions, required arguments, cost limits, and side effects stay in code.
In the catalogue:
— turns tool-selection conditions into structured questions; local code invokes the tool.
3. Model routing
Classify intent and difficulty before choosing a rules engine, small model, frontier model, or human queue. Expensive generation then handles only the requests that need it.
Treat user text such as “route me to the most expensive model” as content to classify, never as a routing command.
In the catalogue:
— classifies request complexity to drive backend model routing;
— classifies a message before model selection.
4. Skill or subagent routing
Choose from a finite skill list and load only the most relevant context. Always include a “none applies” or “stop” option, or the model will be forced to select a bad match.
In the catalogue:
— matches a task brief to candidate rules before local policy picks the execution profile.
5. Confidence-tiered workflows
Automate above a validated threshold, ask for confirmation in a middle band, and stop or escalate below it. Do not copy a threshold from somebody else's blog. Calibrate it against your labeled data, error costs, and the reversibility of the action.
In the catalogue:
— scores destructive actions and scope risk;
— a min_confidence gate in front of live trading entries.
RAG and retrieval
6. RAG passage filtering
Vector search finds candidates. JEV can then judge whether each passage is relevant, trustworthy, contradictory, or contains prompt injection. Only passages that pass policy enter the generation context.
It does not replace retrieval and it does not generate the final response.
In the catalogue:
— judges search intent and the relevance of individual results.
7. Semantic reranking
Use keyword or vector search to shrink the candidate set first. Score the short list with JEV and sort by probability. Do not send thousands of results at once, and never let the model create an item outside the candidate set.
In the catalogue:
— adds an intent judgment after retrieval;
— ranks plausible commands from history candidates.
8. Citation verification
Submit a claim with its source passage and classify the relationship as support, contradiction, or unrelated. Exact quote matching belongs in code. JEV handles the fuzzy question of whether the source actually supports the meaning of the claim.
In the catalogue:
— finds passages matching a field, with quotes and page references; a reviewer verifies every claim.
9. Entity resolution
Use deterministic fields such as names, domains, and addresses to eliminate impossible pairs. Then judge whether two remaining records refer to the same product, company, or person. Keep close calls for human review instead of merging them automatically.
In the catalogue: Nothing close yet. If you have built one, send a PR.
Safety and quality
10. LLM input and output guardrails
Check input before generation and output before delivery. Ask separate questions for jailbreak attempts, harmful content, personal data, and secret leakage so one aggregate score cannot hide the blocking reason.
The model is not the permission system. Allowlists, spending limits, and mandatory approvals still belong in code.
In the catalogue:
and
.
11. A typed alternative to LLM-as-a-Judge
When an evaluation rubric already has fixed levels, another LLM does not need to write a paragraph of commentary. JEV can return a probability for every level, making bulk evaluation easier to rank, threshold, and measure.
Use an LLM or a human when you need an explanation for the score.
In the catalogue:
and
.
12. Semantic code checks
Linters handle syntax and static rules well. They struggle with natural-language constraints such as “does this change violate our architecture rule?” Run a noul judgment over the diff and the rule, and surface a review hint only on a strong match.
This does not replace the compiler, tests, type checking, or static analysis.
In the catalogue:
and
.
13. Compliance checklists
For documents that change frequently, ask a stable set of questions one by one. Obvious passes move forward, edge cases enter a review queue, and the system keeps the original text, model version, probabilities, and final human decision.
In legal or safety-critical settings, the model can provide evidence for a decision but cannot own the decision.
In the catalogue:
.
Data and operations
14. Support-ticket triage
One ticket state can answer several questions in one request: destination queue, urgency, sentiment, and refund intent. Send the state once and write each typed answer directly into the queueing system.
In the catalogue: No direct match yet. The nearest is Tax Document Classifier.
15. Hierarchical classification
Do not place hundreds or thousands of labels into one choice. Select a broad category, then choose within its children. If two branches are close, preserve both candidates for another step or human confirmation.
In the catalogue:
.
16. Candidate-based extraction
Let a parser, regex, or OCR system find exact source spans. JEV chooses which candidate answers the question, and code copies that span verbatim.
JEV selects from evidence here. It does not invent a value missing from the source.
In the catalogue:
.
17. Composite scoring
Avoid a vague question such as “how valuable is this customer?” Score urgency, value, risk, and fit separately, then combine them with application-owned weights. Every dimension can have its own threshold, and the final ordering is easier to inspect.
In the catalogue:
.
18. Semantic features for traditional ML
Use noul probabilities or score outputs as numeric features beside existing structured fields. Measure any lift on a separate validation set and watch for data leakage, drift, and calibration failure.
In the catalogue:
.
When another tool is better
- If only one route is legal, use code.
- If the answer depends entirely on arithmetic, dates, or allowlists, use code.
- If the task must write a reply, plan, program, or open-ended text, use an LLM.
- If the legal answers cannot be listed in advance, use an LLM or a person.
- If identical inputs must always produce identical outputs, use deterministic rules.
- If a high-risk decision needs an accountable owner, keep human approval.
Typed output prevents the response from breaking its declared shape. It does not prevent the model from choosing the wrong legal option. Format validity and judgment accuracy are separate properties.
Six checks before production
- Define the state, legal answers, downstream actions, and the cost of each error.
- Build a labeled set from real traffic, including edge cases, multiple languages, and adversarial inputs.
- Measure per-class recall, calibration, review rate, latency, and cost alongside accuracy.
- Choose thresholds by action risk.
- Log the model version, input, probabilities, selection, fallback path, and final outcome.
- Define fallbacks for timeout, rate limits, low confidence, and model upgrades.
From use cases to source code
The 18 use cases are a design map. The implementation details live in real repositories. Some projects validate the probability distribution before a browser action. Some explicitly defend model routers against instructions embedded in user input. Others make fail-open or fail-closed a visible product decision.
We collected those implementations in
. The catalogue covers 100+ public projects grouped by use case, and every entry keeps its discovery source, repository URL, and a fixed-commit source permalink.
source-reviewed means that we inspected the relevant code at that commit. It does not mean that we ran the project, reproduced a benchmark, completed a security audit, or received an endorsement from its maintainers. This is a traceable developer map, not a leaderboard.
If you already have one bounded judgment to test, the
is a secondary way to run it. On September 20, 2026, our authenticated jev-1.13 request returned HTTP 200, all three typed answer shapes, and usage. That verifies the access path and response contract, not accuracy on your own data.
Start by finding the closest implementation pattern in the source, then decide whether JEV belongs in your system.
Explore 100+ projects in Awesome JEV
Prepared by the BeatAPI team. JEV is developed by TypeSafe AI.
Top comments (0)