There's something awkward about how large models make decisions today: they generate the decision one token at a time.
Ask "should I click this button?" and it streams "you should click…" word by word before giving you the answer. Two problems: slow (every decision runs the full generation pipeline) and expensive (a yes/no burns a pile of tokens).
Laya (10751 stars in 4 days, Apache-2.0, Python) argues it shouldn't work this way. Its pitch: a multilingual, non-autoregressive System 1 decision engine.
How it fixes "slow decisions"
Laya's logic is blunt: decisions don't need "generation", they need "judgment".
It's non-autoregressive — instead of emitting token by token like a normal LLM, it produces the decision directly in a single forward pass. The key numbers:
- 33 milliseconds per decision (an order of magnitude over token generation);
- 100+ languages with typed decisions (structured, typed results, not vague yes/no);
- RLCD training against strictly proper scoring rules, not hand-labeled preferences.
In one line: separate "thinking" from "deciding", and use a small, fast model dedicated to the deciding.
Three signals it's worth watching
- It hits the real agent pain point — decision latency. Agent slowness usually isn't "thinking", it's "waiting for the model to generate at every step." A 33ms decision engine raises agent responsiveness by an order of magnitude — a hard requirement for real-time interaction (browser operation, gaming, test automation).
- "Non-autoregressive + typed decisions" is a genuine engineering insight. A decision is fundamentally "pick one from a finite set" — it never needed token generation. Laya gets this right by defining the problem correctly, not by throwing compute at it.
- It rides one of the hottest narratives this year. The author notes he built decision models a year ago before a frontier lab renamed the same idea. Whatever the claim, "decision models" are among the hottest directions in the agent space right now, and Laya is a serious, usable open-source implementation.
The honest caveat
It's a component, not a full agent. Laya does one thing — decide — and your agent has to wire its own inputs and consume the decision. It doesn't replace your main model; it pairs with it: the main model understands and generates, Laya decides fast. It's also very new (4 days), with API and weights iterating quickly — for research and pre-study, not production.
I've localized the README and docs to Chinese: https://github.com/yangshun2005/laya-cn
If you find this project useful, a star on the original repo supports the author's ongoing maintenance.
Top comments (0)