If you've ever used a large language model to answer a question like "which department should this ticket go to?" or "is this email urgent?", you've probably run into the mismatch. You want a single, typed answer, but the model generates text token by token, and you have to parse it and hope it doesn't wander off-script or invent a label that doesn't exist.
Laya, an open-source project by Nandakishor M under Convai Innovations, is built to fix that directly.
This is a fast-moving story. As of September 24, 2026, the whole "typed decision model" category is about nine days old, and Laya's PyPI history shows 13 releases in its first two days, then roughly one a day. The numbers below come from the project's documentation and independent coverage at the time of writing, so check the repo for anything that has changed since.
The backstory: a gated launch and a three-day open answer
Laya's README benchmarks itself against one rival again and again: Jev, from a startup called TypeSafe AI. Knowing that rivalry helps explain why Laya reads the way it does.
TypeSafe AI was co-founded in 2024 by Diogo Almeida (previously part of the RLHF work behind ChatGPT at OpenAI), Erik Gafni and Sasha Sheng. It came out of stealth on September 15, 2026 with Jev.
Press coverage described Jev as the first "System One Model": a "frontier-class" model of undisclosed size, described as transformer-based, that returns typed, schema-constrained decisions with calibrated probabilities instead of generated text. Launch coverage tied it to a $40 million seed round led by DCVC.
Jev itself is closed: hosted only, with no published weights, no disclosed parameter count and no self-hosting option. Access has shifted quickly. It launched as a waitlisted early-access API, and within days it was also available through Vercel AI Gateway, which doesn't go through TypeSafe's waitlist. Secondary sources also report that TypeSafe opened signups to everyone on September 20, then paused new signups on September 22 because of demand. Check console.typesafe.ai for the current state, since it may change again.
TypeSafe publishes its own comparison numbers at evals.typesafe.ai, and independent reproductions of its headline claims are hard to find, so those claims are difficult to verify from the outside. Its launch post claims Jev is 40x–200x faster than frontier LLMs, and its homepage headlines up to 193.6x faster and 444.6x cheaper, which TypeSafe describes as the high end of real-world gains. Pricing is $0.042 per million input tokens with free output.
Even on TypeSafe's own board, Jev isn't the top scorer on raw workflow accuracy. Its 67.8% is effectively tied with GPT-5.6 Terra at 67.9%, the model TypeSafe itself calls the most comparable, while GPT-5.6 Sol (74.1%) and Claude Opus 5 (73.1%) sit clearly above it. So the real pitch is closer to mid-tier frontier-model accuracy at a fraction of the cost and latency than to leading on accuracy.
Even so, the launch got enough attention to be covered by The Register, MarkTechPost and LangChain's own blog.
Three days later, on September 18, 2026, Nandakishor M published Laya along with a dev.to write-up titled, in part, "I Built Non-Autoregressive Decision Models a Year Ago." The post argues that a well-funded newcomer was getting credit for an idea he had already built.
Laya is pitched as a direct answer to Jev: an Apache 2.0, self-hostable model family built on the same basic idea (typed decisions from a single forward pass), but shipped with published weights, code and benchmarks.
There's some related prior work behind that claim. Nandakishor M has an earlier arXiv paper, SalesRLAgent (submitted March 2025), on using specialized reinforcement learning instead of autoregressive LLMs to predict sales-conversion probability in real time. By the dev.to post's own account, that 2025 system used frozen embeddings plus PPO for sales conversations and couldn't handle new questions at runtime, and Laya is described as a rebuild.
The dev.to post also cites a second paper, from September 2025 (arXiv:2510.01237), as laying out the framework for schema-based decisions guided by reinforcement learning. Its abstract, which is as far as this article has checked, describes confidence-aware routing to reduce LLM hallucination. It mentions neither RL nor typed decisions. Readers weighing the "I built it a year ago" claim should read that paper and judge for themselves how closely it maps to what Laya does.
So there is prior related work, but a year-old version of this model family isn't demonstrated, and none of it rules out that Laya was put together quickly once Jev created an opening.
At least one independent write-up, from Wavect, made exactly that distinction. It concluded Laya is "a credible open-source decision-model candidate, not a proven universal Jev replacement." That's about the right level of confidence to start from.
The core idea: skip the generation
Laya calls itself a non-autoregressive "System 1" decision engine, borrowing the same psychology framing TypeSafe uses for Jev: fast, intuitive judgment (System 1) versus slow, deliberate reasoning (System 2).
Instead of generating text one token at a time, Laya's encoder-based models read a piece of text once (a JSON document, an email, a support ticket) and produce typed answers in a single forward pass.
There's no decoding loop, so there's nothing to parse and no way to invent a label that doesn't exist. You get a structured answer back directly. (It can still be wrong, as the Khmer example below shows.)
The project claims about 33 milliseconds for a single question on the multilingual checkpoint (39.5 ms for English), dropping to roughly 7.2 ms per question when batched, measured on a T4 GPU.
Three decision primitives
Laya sorts every question into one of three typed formats:
choice: a labeled classification (which department, which intent, which category), returning the top label, a probability for every option, and a confidence score.score: a value on an ordinal rubric (e.g., urgency from "not urgent" to "critical"), with a distribution and confidence.noul: a calibrated true/false probability from 0.0 to 1.0, useful for things like churn-risk detection or spam filtering.
You define the questions declaratively, alongside the state you want evaluated, and the model answers all of them in one pass:
questions = {
"department": {
"type": "choice",
"instructions": "Which department should handle this request?",
"criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages"}
},
"urgency": {"type": "score", "instructions": "How urgent is this?",
"criteria": ["not urgent", "soon", "critical"]},
"churn_risk": {"type": "noul", "instructions": "Does the user threaten to leave?"}
}
Three checkpoints, one router
Laya ships three checkpoints with different tradeoffs:
| Checkpoint | Encoder | Params | Context | Best for |
|---|---|---|---|---|
| laya | ModernBERT-large | 421M | 512 | English |
| laya-multilingual | mmBERT-base | 322M | 1024 | 100+ languages, faster |
| laya-typed-decisions | ModernBERT-large | 421M | 1024 | Fine-tuned typed-decision workflows (selected explicitly) |
The Router picks between the English and multilingual checkpoints automatically, detecting script in under 0.5 ms and language in under 1 ms. The typed-decisions checkpoint isn't selected by default. You choose it with an explicit model= override, or by turning on the opt-in auto_task_detection.
Routing matters because on non-Latin scripts the English-only checkpoint doesn't just perform worse, it stays confidently wrong. The project's own benchmarks show it scoring 0% accuracy on Khmer while reporting 95%+ confidence, which is exactly the failure mode that makes naive confidence-gating dangerous.
The router is meant to sidestep this by choosing the checkpoint before the forward pass, using script and language heuristics rather than model confidence. It isn't foolproof. The README calls its language detector a best-effort heuristic on Latin-script text that defaults to English for short input, and its own example sends a Romanian sentence to the English checkpoint. Script detection (Khmer, Devanagari and so on) is the more reliable half.
In production you can preload the checkpoints into memory (Router(preload=True)) so language switches never trigger a reload, or load them lazily and let LRU eviction keep recently used ones resident.
How it stacks up against Jev, on paper
Laya's documentation keeps returning to a head-to-head comparison with Jev, and Laya's self-hosted server is designed to be wire-compatible with it. Laya reports being roughly 6–7x faster per question, with no per-token fees when self-hosted (you still pay for the compute), along with higher accuracy on most shared benchmarks (AG News, DAIR Emotion, typed-decisions).
On DAIR Emotion, the docs claim Jev assigned zero probability to the correct label on 16% of examples, a real problem for any pipeline that branches on confidence.
Before taking those numbers at face value, though, it helps to know what they are. Laya's GitHub README says up front that every Jev figure in its tables is a third-party published number, not something measured against a live Jev API, since TypeSafe hasn't granted outside benchmark access. So "Laya vs. Jev" comparisons, including the ones in this article, are really "Laya, measured directly" versus "Jev, as reported by others."
That isn't a controlled, identical-conditions test, and the README says so rather than burying it. The speed comparison has another wrinkle: it puts Laya on a local GPU against latency figures from third-party benchmarks of Jev's hosted API, so the hardware isn't like for like.
The comparison isn't one-sided, though. On high-cardinality label spaces (Banking77's intents), Laya's default token budget for options causes accuracy to collapse to 0.425 versus Jev's reported 0.870, because each label only gets a handful of tokens to stay distinguishable. Even that comparison isn't clean: Laya's run uses 77 labels, while Jev's reported setup uses 72.
Laya also trails on "soft" distribution-matching against a teacher model's probability spread, and on raw calibration before temperature fitting.
What independent testing found
Both models are new enough that outside scrutiny is still catching up, so it's worth looking past each project's own materials. A community-run comparison published as a Hugging Face dataset (Luni/laya-jev-benchmark) ran Laya against Jev's published numbers. Its phishing-detection task is one neither model was trained on, and the results there are less flattering than the headline tables above.
On a 2,000-email phishing set, Laya's raw, out-of-the-box accuracy was 0.505, essentially chance. It predicted "not phishing" for almost everything, with a recall of just 1.2%.
Its ranking ability was another story. AUROC was 0.678, close to Jev's reported 0.689, so the signal was there. The problem was an uncalibrated decision threshold rather than the model's judgment.
Fitting a bias term (Platt scaling) on held-out data brought accuracy up to 0.611. That's still short of Jev's reported 0.626, though the two aren't like for like: per the Luni write-up, Jev's 0.626 is a raw score, while Laya's 0.611 uses a calibration half of the data. It's also well behind Claude Haiku 4.5, which the same evaluation reports at 0.813, though at roughly 76x the latency (687 ms vs. Laya's 9 ms). Haiku's latency is a published figure, not one measured on the same hardware, and Laya's 9 ms was measured on an RTX 5090, a much faster GPU than the T4 behind the latency figures earlier in this article.
The same evaluation also ran Laya through a set of consistency probes written by its author. Laya failed 7 of 11, and its verdicts shifted when the option names changed. The probe suite is self-written and small, so treat that as a warning sign rather than a formal result.
The write-up also pushed back on an "83.8% vs. 67.8%, +16.0% advantage" comparison, pointing out that the two numbers came from different benchmarks entirely, which makes the comparison meaningless.
That claim appears in the table of Nandakishor M's dev.to post. The Luni author attributes it to Laya's model card, but it isn't in the current README or model card, so it may have been dropped since.
The 67.8% figure appears to be Jev's score on TypeSafe's own workflow-accuracy eval board, where "correct" means the consensus of two other frontier models rather than human-verified ground truth. That's a second layer of apples-to-oranges on top of the first.
(The GitHub README's own comparison table does compare same-benchmark rows for both models.) It's a good reminder to check what's actually being measured before trusting any single comparison number.
The practical takeaway is close to what Laya's own "Honest limits" section says. The raw checkpoints are not zero-shot-ready, calibration work matters a lot, and a general-purpose LLM can still beat a specialized typed-decision model on tasks outside its training distribution, just much more slowly and expensively.
Getting it running
Laya needs Python 3.10+ and installs from PyPI:
pip install laya
There's also a CLI for quick checks. laya "I was charged twice, please refund" runs routing only (no model download, works offline), and adding --predict loads the actual checkpoint.
For serving, pip install "laya[serve]" gives you laya-serve, a FastAPI server that speaks the same POST /v1/systemone protocol as the hosted Jev API, so existing Jev clients can often just repoint their base URL. One security note: per the model card, laya-serve binds to 0.0.0.0 with no authentication unless you set LAYA_API_KEY, so set a key (or put it behind a proxy) before exposing it beyond your own machine.
For agent tooling, there's an optional MCP stdio server (laya-mcp-server) that exposes laya_predict, laya_route, laya_preset and laya_status as tools for MCP-compatible clients like Claude Desktop or Cursor.
There are also LangChain/LangGraph integrations (LayaRouter, LayaGuardrail) for dropping typed decisions into an agent graph as conditional edges or guardrails.
A few other engineering details:
An optional TileLang-based GPU fast path fuses kernels and uses CUDA graphs per shape bucket.
Batch scoring (
predict_batch) packs many states into shared forward passes for a roughly 9–10x GPU throughput gain, measured on an RTX 5060 Ti rather than the T4 behind the latency figures above.In the recent 0.3.x releases, checkpoint loading dropped from ~22s to ~2s on CPU, and
import layano longer pulls in torch until a model is actually used.
Built for fine-tuning, not just zero-shot
The docs are candid about one thing: the base checkpoints do poorly on Laya's own typed-decisions benchmark when used zero-shot. They score 0.36, barely above the 0.318 random baseline and below the 0.461 a majority-class guesser would get. The real capability comes from fine-tuning.
A provided notebook runs the full loop on free dual-T4 Kaggle GPUs in about 4–5 hours: dataset construction, RLCD training via GRPO-style policy gradient against proper scoring rules, calibration fitting, and pushing to the Hugging Face Hub.
After fine-tuning, the README reports the same benchmark jumping to 0.766 with the laya-typed-decisions checkpoint (the one you have to select explicitly). That's ahead of Jev's published number and above the teacher "self-agreement ceiling."
That last part deserves some caution. The Luni benchmark lists a near-identical 0.767, but that row is Laya fine-tuned on the task itself, so it isn't an independent replication on unseen data.
Its author warns that beating the teacher ceiling (0.735 there) suggests the model memorised label noise rather than understanding the task better. So 0.766 is evidence that fine-tuning works, not proof of a new accuracy ceiling.
A separate worked example fine-tunes Laya as a browser-automation decision head, reportedly taking element-selection accuracy from 10% to 66% and task success from 0% to 62%.
The limitations section is worth reading
Most project READMEs bury their caveats or skip them. Laya's has a dedicated and fairly long "Honest limits" section, and it's useful:
Boolean-word labels like
true/falseinchoicequestions can get echoed back regardless of meaning.The
noulprimitive can latch onto its own option labels rather than the actual text, especially on the English checkpoint.The multilingual checkpoint has a documented position bias on ordinal
scorequestions.The
action.act_probabilityfield is flagged as carrying no real signal yet.
Taken together with the phishing evaluation above, the pattern is clear. This is a strong, fast base model that needs calibration and task-specific fine-tuning before you trust its confidence scores in production. It isn't a drop-in zero-shot replacement for a bigger model.
Bottom line
Laya is aimed at a specific, common problem: using encoder models instead of generative LLMs for the classify, score and yes/no decisions that show up constantly in agent pipelines, ticket triage, moderation and routing logic.
It exists in its current form largely because a well-funded, closed competitor forced the comparison into the open.
It's Apache 2.0 licensed and has PyPI and Hugging Face releases, a TypeScript port for Node and the browser, Docker and Nix packaging, and a few community add-ons already building on it (an MCP judge plugin, Google ADK tools and an Ascend NPU port).
The other side is moving too. MarkTechPost's coverage of Nokia's AnyJev reports a calibration error (ECE) of 0.036 versus Jev's 0.144 on a typed-decisions set, though fine-tuned Laya still leads on accuracy there.
The self-reported benchmarks are favorable, the documented limitations are frank, and the early independent testing is mixed: encouraging on ranking ability, less so on out-of-the-box calibration.
Both the project and its category are very young, so it's best to hold both loosely. The architecture looks promising, but the maturity is still unproven. Test it on your own data before gating anything important on its confidence scores.
Sources
Laya
Independent evaluation and commentary
Jev and TypeSafe AI
Published via ZyVOP — Write once in Markdown, auto-backup to GitHub, and syndicate to Dev.to, Medium & Hashnode in 1 click.
Top comments (0)