I built four AI apps on Laya, and every one of them runs entirely inside a browser tab: no server, no API key, no cost per call.
An AI agent. An LLM-as-a-judge for RAG. Agentic workflow automation. Long-term agent memory that knows when to forget. All four run on one 421M-parameter model using WebGPU, and on the job that matters most for agents (deciding what to do next) it beat a generative LLM nearly four times its size.
Here's what I built, the numbers, and the design patterns that make it work.
What is Laya?
Laya is a 421M-parameter decision model from ConvAI Innovations, built on ModernBERT-large and open source under Apache 2.0. It doesn't generate text. You give it a typed question (pick one of these options, yes or no, or a score on a scale) plus some context, and in a single forward pass it returns a probability for every option.
That makes it a non-autoregressive "System 1" model: about 33–40 ms per decision on a T4 GPU, versus 236–276 ms for TypeSafe's generative Jev (AI Weekly). It's built for the high-volume decisions every AI system makes: ticket routing, content moderation, AI guardrails, agent-trace checks.
"System 1" is Kahneman's fast, intuitive thinking. The idea is that most of what an AI agent does all day (which tool, which queue, is this safe, are we done) is System 1 work, and you shouldn't be paying a frontier LLM to do it.
This is one of the big themes of 2026. NVIDIA researchers argued that small language models are the future of agentic AI, and the industry is moving towards small models handling routine agent work while big models take only the hard cases (a good roundup). At the same time WebGPU has made running real AI models in a browser tab a practical, production-ready choice.
Laya sits right where those two trends meet. So I wanted to see: how much of an AI stack can a 421M decision model run, on a laptop, in a tab?
Getting Laya into the browser
I converted the typed-decisions checkpoint to ONNX and quantized it (layaForWeb, model on Hugging Face as VishalMysore/layaForWebTrained):
- int8: about 422 MB, runs on WASM
- int4: about 278 MB, runs on WebGPU
It downloads once from Hugging Face, is cached in the browser's Cache Storage, and runs through ONNX Runtime Web. GitHub Pages can't send the COOP/COEP headers that multithreaded WASM needs, so each app registers a tiny service worker that adds them. It's fully private, local AI: nothing you type leaves the page.
One trick I used in every app: playback before download. Each page ships answers Laya gave for the built-in examples and plays them back (clearly labelled) until you load the model. The demo works in one second, and the live model is one click away.
Then I built four apps on top of it.
1. layaForWorkflows: agentic workflow automation as a graph
A workflow is just data: a DAG of three node types. Diamonds are typed questions Laya answers. Boxes are automation steps. Circles at the end say whether the case was handled automatically, sent to a person, or blocked.
Type a support ticket ("the app closes as soon as I open it, client demo in an hour") and the path lights up with Laya's probability on every branch: bug, 78%, then "is the customer blocked right now?" yes, 67%, so it pages the on-call engineer.
Seven workflows ship with it, including support ticket triage, IT incident response, insurance claim intake, content moderation and an AI agent action guardrail that fails closed: deletions are blocked, and anything uncertain waits for a human.
Laya answers every decision node in one batched forward pass, including the ones off the path the run took. So the confidence slider re-routes a finished run instantly, with no model call, and the Analytics tab shows how the mix of automated, human-reviewed and blocked cases shifts as you raise the bar.
Across the 37 example inputs, Laya sent the routine cases straight through, routed the risky ones to people and blocked the dangerous ones, with each decision drawn on the graph and exportable as an OWL ontology in Turtle or JSON-LD. A force-push to main correctly went to a human for approval. That's exactly the behaviour you want from an AI guardrail.
2. layaAsRagJudge: LLM-as-a-judge without the LLM
LLM-as-a-judge is everywhere right now, usually meaning another expensive API call to a big model to check the first one. Laya does the core of that job in the browser: is this claim actually supported by the retrieved evidence?
The whole RAG pipeline runs in the tab: sentence-aware chunking, all-MiniLM-L6-v2 embeddings, an IndexedDB vector store, cosine top-3 retrieval, and then one typed question to Laya. The verdict goes through a gate: AUTO, BLOCK, or HOLD for review.
On 72 claims about fictional documents (so the model can't lean on what it already knows):
| Judge question | Accuracy | False "supported" | AUROC |
|---|---|---|---|
| Two options (default) | 75.0% | 27.8% | 0.80 |
| Three options (supported / contradicted / not mentioned) | 72.2% | 11.1% | 0.80 |
| Yes / no | 69.4% | 19.4% | 0.74 |
That's with no fine-tuning for this task, just a typed question. With the confidence gate on, Laya acts on its own on 44% of claims at 84% accuracy and flags the rest for review. Switch to the three-option question and false "supported" verdicts drop to 11%.
Then I built a RAG router on top: Laya decides which knowledge base (Encyclopedia, News or Policy) should answer a question, or whether to skip retrieval entirely. It picked the right knowledge base 23 times out of 24. With a second Laya question ("do these passages answer it?") checking the results, the router got the right final outcome on 94% of questions, all in the browser.
3. layaAgent: a 421M encoder beat a 1.5B LLM at running an AI agent
This is the result that surprised me most.
layaAgent is an AI agent that runs entirely in a browser tab. Laya is System 1: at each step it picks the next tool from 12 options and picks each argument from candidate spans. A small LLM running in the tab through WebLLM (Qwen2.5-1.5B-Instruct) is System 2, a generative fallback. Tools call Wikipedia, Wikidata and Open-Meteo straight from the browser, plus a calculator, unit and date math, and notes stored in IndexedDB.
The rule that makes this work: Laya never has to write. It doesn't type "Japan"; it picks "Japan" out of the spans an extractor found in the last tool result. Tool calling becomes a choice, and choices are what Laya does best.
I expected the hybrid (Laya first, Qwen when Laya is unsure) to win. It didn't. Laya did.
On its own, Laya more than tripled the task success of the 1.5B LLM (32% vs 10%), and ran each task in under a third of the time (9.7 s vs 35.3 s). Handing Laya's uncertain steps to the LLM actually made things worse. Picking from options is what Laya was trained for, and it shows.
Even better, Laya knows when it knows. On unseen tasks, the steps Laya took on its own were right 90% of the time, and everything else was visibly held with a reason. Put a person on those held steps (human-in-the-loop) and 94% of tasks finish correctly.
On top of that, anything that changes something (note_save, note_delete, send_message) always asks you first, no matter who picked it:
Laya is especially strong on single-tool goals (weather, arithmetic, unit conversion, dates, memory) and at spotting risk: its "does this goal change something?" check scored an AUROC of 0.90.
4. layaForMemory: AI agent memory that knows when to forget
Agent memory is one of the hottest problems in agentic AI, and the hardest part is that facts go stale. "We use Postgres" stops being true the day the migration lands, and your assistant keeps repeating it.
layaForMemory is a memory invalidation layer, following the design of invalidate and rebuilt for the browser. Every event (a chat message, a settings change, a Slack-style note) is checked against similar stored facts, and Laya decides what it does to each one: confirmed, superseded, contradicted, needs review, or unrelated. Memory text is never edited. Stale facts are struck through and linked to their replacement, and every verdict is written to an audit ledger before memory changes.
It also has built-in prompt-injection defence, and it's beautifully simple. Before touching memory, Laya answers one question: is this event a report, a question, a proposal, or a command? Commands never change memory.
On 62 test cases:
- Zero good facts wrongly thrown away (0 of 32)
- 100% of questions, plans and prompt-injection commands recognised and ignored
- Combining four Laya questions separated stale from current facts with an AUROC of 0.95
7 design patterns for building on Laya
These held across all four projects.
1. Gate on branch probability.
Use the share of Laya's probability that went down the chosen route as your confidence gate. One slider then means the same thing on every decision node: "act on your own only if at least X of the probability agrees". It's simple and it works everywhere.
2. Choice questions are Laya's superpower.
The 4-way "what kind of message is this?" separated prompt-injection commands from everything else perfectly on my sample. The router picked the right knowledge base 23 of 24 times. Whenever you can phrase a decision as "pick one of these", do.
3. Route on what something is.
Laya is at its best classifying intent, category and risk. Design your questions around what a ticket, claim or event is, and let plain code handle the arithmetic.
4. Wording is a hyperparameter.
"The user is now pescatarian instead of vegetarian" cleanly superseded "The user is vegetarian". Phrase events as direct statements and test your question wording against real model outputs. A little prompt tuning goes a long way.
5. Ask several questions and average them.
Each staleness question scored an AUROC of 0.89–0.92 on its own; their mean hit 0.95. Laya is fast enough that four questions in one batch cost almost nothing.
6. You may not need a generative fallback at all.
In my agent, Laya plus a person beat Laya plus an LLM by a wide margin. Sometimes the best System 2 is a human.
7. Record once, re-score forever.
Every app stores Laya's full answers and lets sliders re-route them without another forward pass. Threshold tuning becomes dragging a slider, and the trade-offs are visible to anyone opening the page.
Can a 421M model run your AI agents?
The decision layer, yes. And that's most of what an agent does. Build around what Laya is great at:
- Let code propose the options. Let Laya pick.
- Gate on branch probability, and make "I'm not sure" a first-class outcome.
- Keep side effects behind a confirmation no model can skip.
- Run it locally with WebGPU: no API key, no server, no cost per call.
All four apps are open source (Apache-2.0) and run on GitHub Pages. Open one, hit Run, and watch the decisions light up before you've downloaded a single megabyte:
- 🔀 Workflows: https://vishalmysore.github.io/layaForWorkflows/
- ⚖️ RAG judge + router: https://vishalmysore.github.io/layaAsRagJudge/
- 🤖 Agent: https://vishalmysore.github.io/layaAgent/
- 🧠 Memory: https://vishalmysore.github.io/layaForMemory/
If you're building AI agents with small language models, I'd love to hear: which part of your agent do you think actually needs a big LLM? Tell me in the comments.
Results come from the evaluation pages in each repo, on the test suites shipped there. This is an independent project, not affiliated with ConvAI Innovations. Laya is by ConvAI Innovations (Apache-2.0), built on ModernBERT-large by Answer.AI and LightOn.
Top comments (0)