Convai Innovations has released Laya, a model that answers typed questions instead of writing text, under the Apache 2.0 licence. It holds 421 million parameters and can be downloaded and run locally. That is the difference that matters, because the commercial model it follows is available only through a waitlist.
Laya takes the same shape of work this site covered when TypeSafe's Jev returned typed decisions rather than text on September 15. You hand the model a state, such as an email or a JSON document, plus questions with defined answer types. It fills in every answer in one pass and attaches a confidence number to each.
What is actually in the box
The model card describes a small stack rather than a new architecture. ModernBERT-large supplies 395 million of the parameters. On top sits a decision head trained from scratch: two transformer layers, a scorer for answer options, and a head that chooses between acting and escalating.
The English checkpoint reads up to 512 tokens at a time. A multilingual variant uses a different base, mmBERT-base, holds 322 million parameters, reads 1,024 tokens and covers more than 100 languages. The model is installed as a Python library.
A 512-token window is short. That is roughly one email, not one support thread, and it is the first limit to test against your own inputs.
The speed claim, and who measured it
The project site reports a median latency of 32.8 milliseconds for a single question on a Tesla T4 graphics card. Batched ten at a time, it reports 7.2 milliseconds per question.
The comparison figure is where care is needed. The model card puts Jev at 236 to 276 milliseconds and calls that a 7.8 times speedup. It also says plainly that the "Jev figures are third-party published, never measured here."
The accounts also differ on which checkpoint hit 32.8 milliseconds. AI Weekly reports that figure for the multilingual variant and 39.5 milliseconds for the English one, while the project site presents 32.8 milliseconds as the headline. Both are on the same class of hardware.
The accuracy numbers need reading twice
Laya's own materials are unusually candid about what its headline score means.
| Measurement | Value |
|---|---|
| Typed decisions, fine-tuned on that benchmark's training split | 0.766 |
| Typed decisions, zero-shot | 0.362 |
| Random baseline on the same benchmark | 0.318 |
| Majority-class baseline | 0.461 |
| Email spam filtering | 0.993 |
| Phishing detection | 0.980 |
| Banking77 intent | 0.425 |
The second row is the one to sit with. Zero-shot, the model scores 0.362 where guessing scores 0.318 and always picking the most common answer scores 0.461. The model card states the conclusion itself: "Laya is a fast base to specialise, not a zero-shot decision engine."
Calibration follows the same pattern. Confidence numbers are meant to match real accuracy, and the reported expected calibration error starts at 0.466. It reaches the advertised 0.081 only after temperature refitting for each question type, which is a tuning step you perform, not a property you inherit.
Founder Nandakishor Mukkunnoth names one more limit: "Choice questions degrade with >20 options."
What this means for developers
Treat this as a base to fine-tune, not a drop-in decision service. The gap between 0.362 and 0.766 is entirely the training split of one benchmark. If you cannot label a few thousand examples of your own decision, the zero-shot figure is the one that predicts your results.
Where it does fit, the economics change completely. Spam and phishing classification score above 0.98, those are exactly the narrow binary decisions the design targets, and running locally removes the per-token bill and the waitlist. A routing step that costs tens of milliseconds on a T4 can sit inside a request path, which an API call to a frontier model cannot.
Budget for the calibration work before you trust a confidence number. Plan to refit temperature per question type on your own data, then log confidence against outcomes for a week and check whether 0.7 really means seventy percent. This is the same verification this site suggested for Jev, and an open checkpoint at least lets you do it yourself.
Check the licence against your plans while you are there. Apache 2.0 permits commercial use and modification, so the honest comparison with a closed API is not only latency but who can audit the thing when a decision goes wrong.
This article was first published on Tech AI Wire.
Also available in
Deutsch · 日本語 · Français · Español · Português
Related on Tech AI Wire
Sources
- Laya - 33ms Multilingual System 1 Decision Engine with Calibrated Probabilities - Convai Innovations
- convaiinnovations/laya model card - Hugging Face
- Convai ships Laya, a 421M ModernBERT decision model, Apache 2.0 - AI Weekly
Top comments (0)