Fast Decisions in Agent Workflows: Laya vs TypeSafe Jev

Originally published on JevLab. This is an independent, unofficial evaluation write-up — not affiliated with TypeSafe or Convai Innovations.

TL;DR: Using a 70B autoregressive LLM for simple agent routing is increasingly the wrong default. Convai Innovations' open-source Laya (421M ModernBERT-large) challenges TypeSafe's hosted Jev primitive. Published charts put Laya roughly 7.8× faster (32.8 ms vs 236–276 ms), with tighter calibration after temperature refit (ECE 0.081 vs 0.246), plus broad multilingual reach at self-hosted software cost of $0. But in production agent pipelines, raw accuracy is only half the story: confidence thresholds and fail-closed handoffs decide whether a wrong route ever executes.

1. The "breakthrough" debate and System 1 decision engines

In early September 2026, TypeSafe launched its proprietary Jev engine (e.g. jev-1.13.0) and framed non-autoregressive decision primitives as a breakthrough for AI workflows. Instead of generating variable-length text, Jev returns a deterministic choice plus a confidence score.

Shortly afterward, Nandha Kishor M (Convai Innovations) published a widely shared DEV post: I Built Non-Autoregressive Decision Models a Year Ago. Then a Frontier Lab Called It a "Breakthrough". Convai also released Laya — Apache-2.0, ModernBERT-large weights on Hugging Face (convaiinnovations/laya).

(If you have the exact DEV URL for Nandha's post, swap it into the link above before publishing.)

This is an architectural fork in agent engineering: System 1 (fast, non-generative classification) vs System 2 (slow, autoregressive reasoning).

Incoming request
        │
[System 1: fast decision engine]
(Laya ~33ms local / Jev hosted API)
        │
Confidence >= threshold?
   /              \
 YES               NO
  │                 │
Auto-adopt route   Fail-closed handoff
(specialist tool)  (human / fallback)

Using GPT-4 / Claude-class models just to pick among five tools often costs hundreds of milliseconds to multiple seconds, burns tokens, and risks schema/JSON failures. Decision engines skip text decoding: one forward pass → categorical log-probabilities.

2. Head-to-head numbers (with an important caveat)

Convai published a multi-axis comparison of Laya against third-party published TypeSafe Jev figures (e.g. work attributed to AbdelStark and nlbzard). That means useful directional signal — not a byte-identical side-by-side under identical network conditions.

Metric	TypeSafe Jev (1.13.0 published)	Laya (Convai)	Delta
P50 latency (1 q)	236–276 ms (hosted API)	32.8 ms (local T4)	~7.8× faster
Batched (50 q)	N/A (concurrency capped)	7.2 ms / q (337 ms total)	local scale
Calibration (mean ECE)	0.246	0.081 (temp refit)	~3× tighter
License / price	Closed API (~$0.042 / 1M tokens)	Apache 2.0 ($0 self-hosted software)	open
Multilingual	Unspecified / EN-focused	45 / 51 languages usable	broad
Privacy	Public endpoint egress	Air-gapped / local	zero egress

Accuracy on public sets (as reported)

• typed-decisions (2,000 evals): Laya 0.766 vs Jev 0.727 (+3.9%); Laya above estimated teacher ceiling (~0.735)
• AG News (4 labels): 0.950 vs 0.910 (+4.0%)
• DAIR Emotion (6 labels): 0.595 vs 0.480 (+11.5%)

Held-out workflow-style tasks (Laya checkpoints)

• Phishing: 0.940–0.993
• Email spam: 0.958–0.993
• Topic categorization: 0.930–0.953
• Jailbreak / guardrails: 0.708–0.762
• RAG passage relevance: 0.625–0.657
• 10-way support triage: 0.502–0.522

3. Why Laya can be ~33 ms: ModernBERT + calibration training

ModernBERT-large (421M)

Encoder-style stack (FlashAttention-2, RoPE, unpadding, larger context) evaluates tokens in parallel. That avoids the O(N) decode loop of autoregressive routers.

RLCD-style calibrated training

Classic classifiers overfit to top-1 argmax and become overconfident. Laya-style training pushes toward proper scoring (Brier / log-loss), so probabilities are more usable as policy inputs, not just leaderboard cosmetics.

Checkpoint specialization + preload

from laya import Router

router = Router(preload=True)

decision = router.route({
    "task": "Review pull request #104 for potential race conditions",
    "context": "diff --git a/worker.go b/worker.go..."
})

Mixed-language workloads that reload weights pay a large cold-load penalty (Convai cites ~7.4s). Preloading keeps per-call latency flatter across languages.

4. The part that actually ships: ECE + fail-closed routing

In agents, calibration beats raw accuracy.

• Predict code_search at 0.98 → auto-adopt is reasonable.
• Predict code_search at 0.52 → you are guessing; fail-closed handoff (human review or a clarifying question) is the safe policy.

ECE (lower is better)

Laya (shipped raw):     ~0.466
Jev (published):        ~0.246
Laya-Multi (refit):     ~0.106
Laya (temp refit):      ~0.081

Uncalibrated models can say "95% confident" on inputs they get wrong ~40% of the time. Temperature / domain refit is not optional trivia — it is how you make thresholds meaningful.

5. JevLab's angle: measure the handoff boundary

Our working thesis:

Accuracy without thresholding is an illusion. In production, your policy lives or dies by the handoff boundary.

Two caveats we keep repeating:

1. Third-party tables ≠ controlled A/B. Prefer identical prompts, identical network/runtime assumptions, and pinned model versions.
2. Ambiguity is the real production tax.
   • "Find where the auth token is parsed and update the tests to mock it" → multi-intent (code_search + test_runner)
   • "What does this function do?" with no code attached → information-deficient

An unconstrained router will still emit a label with inflated confidence and corrupt downstream state. Thresholding exists to turn "I'm guessing" into handoff, not into a wrong tool call.

How we structure evals conceptually:

• Clear validation/test sets → auto-adopt rate and error among auto-adopted decisions
• Adversarial / ambiguous challenge sets → does confidence collapse so a threshold like t = 0.80 can trigger handoff cleanly?

6. Production decision matrix

Prefer Laya if you need:

• sub-~50 ms P50 locally
• air-gap / no egress
• existing GPU (T4/L4-class) and ops willingness
• strong multilingual coverage

Prefer TypeSafe Jev if you need:

• serverless / zero GPU ops
• pure TypeScript / edge-friendly integration
• low or bursty volume where API cost beats a always-on GPU

Need zero egress / HIPAA / air-gap?
        /                \
      YES                 NO
       │                  │
    Choose LAYA     Have GPU infra?
                     /          \
                   YES           NO
                    │             │
              Need <50ms?     Choose JEV
               /      \       (serverless)
             YES       NO
              │         │
          LAYA     Evaluate both

7. What we're building next

Open decision models like Laya reinforce the bet: System 1 routing is infrastructure, not a demo.

We're evaluating a Laya baseline inside JevLab so builders can:

1. Compare engines on the same agent-routing tasks
2. Sweep confidence thresholds and watch auto-adopt vs handoff trade-offs
3. Export TypeScript-oriented router policy shaped by those curves

Until a published measured report lands, treat the public site as an unofficial threshold lab / preview — useful for thinking in policy space, not as a claim that every chart is a finalized ship gate.

Read the full original + try the lab: https://jevlab.dev/blog/laya-vs-jev · https://jevlab.dev

---

Unofficial independent project. Not affiliated with, endorsed by, or maintained by TypeSafe AI or Convai Innovations. Benchmark figures cited above come from published third-party / vendor materials unless otherwise noted.