DEV Community: Yanng

Open-source Jev alternatives: System One models you can self-host

Yanng — Fri, 25 Sep 2026 12:55:18 +0000

TypeSafe Jev is the hosted model that introduced System One models on September 15, 2026: you send a state and typed questions, and it returns answers with probabilities instead of generated text. It is generally available and billed per token: $0.042 per million input tokens, with output tokens free.

Within days, open-source Jev alternatives appeared that do the same job on your own hardware. This post lists the ones worth testing, which of them accept Jev API requests unchanged, and what it takes to self-host one.

Why look for a Jev alternative?

Data stays in your infrastructure. Tickets, emails and user messages never leave your servers.
Cost at volume. A self-hosted model costs the same whether you make ten decisions or ten million.
No vendor dependency. Open weights keep working if a hosted API changes its pricing, limits or terms.
Fine-tuning. You can train an open model on your own labelled decisions. Jev's weights are not public.
Offline and edge use. Some models run on a laptop CPU or an Apple Silicon Mac.

Best Jev alternatives to try first

If you already call Jev, start with a model that serves the same POST /v1/systemone endpoint, so existing client code only needs a new base URL:

Model	Size	License	Serves the Jev API	Good first choice when
Laya	322M / 421M	Apache 2.0	Yes, through laya-serve	You want a small model that runs on CPU, or 100+ languages
Kev	0.8B to 27B	Apache 2.0	Yes, the TypeSafe SDK works unchanged	You have a GPU and want the closest drop-in
Decider	2B / 4B / 35B MoE	Apache 2.0	Yes	You serve with vLLM and want many-option choices
Von	395M	Apache 2.0	Yes	You need low latency on modest hardware
OpenThai-SystemOne	0.8B	Apache 2.0	Yes	Your input is Thai

Other open options use their own APIs: Bespoke Nimble (9B), GLiNER2.5-Decide (340M), Together AI's Tev1-4B, and the training-free AnyJev and SemIf. See the full System One model comparison for all twelve.

Jev-compatible API servers

These projects put a Jev-style HTTP API in front of an open model, so you can switch models later without touching client code:

Server	Language	Notes
laya-serve	Python	Ships with the upstream Laya package. See Self-host Laya.
Arbiter	Python	Routing, batching, metrics and a built-in playground
sys1	Rust	candle-based, token batching, CPU, CUDA and Metal
ollaya	Rust	Ollama-style CLI and daemon; pulls models by name
laya-server	TypeScript	Docker image with a web console and API keys

For the two Rust servers side by side, see Run Laya in Rust.

How to switch from Jev to a self-hosted model

Pick a model from the table above and start its server locally.
Point your client at it. With the TypeSafe SDK, change the base URL; ollaya, for example, documents TYPESAFE_BASE_URL=http://localhost:11435. With plain HTTP, the request body stays the same:

curl -X POST http://localhost:8080/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state":{"message":"I was charged twice"},"questions":{"refund":{"type":"noul","instructions":"Does the customer ask for a refund?"}}}'

Compare on your own data. Send the same labelled requests to Jev and to the new model, and compare accuracy and latency before moving production traffic.
Recalibrate thresholds. Probabilities from different models are not interchangeable. If you automate a decision above a confidence cut-off, set that cut-off again on held-out data.

Jev vs open alternatives: what you give up

Hosted Jev still leads out of the box on several published comparisons, especially on tasks with many labels. On Banking77, with 77 intents, upstream Laya reports 0.425 at default settings against Jev's 0.870. Open models close much of the gap after fine-tuning on your own data, and they win on cost, privacy and latency when you run them close to your application. See Laya vs Jev for the details and caveats.

Benchmarks in this space are young and mostly self-reported on different datasets, so treat any single number as a starting point, not a verdict.

FAQ

Is there an open-source version of Jev?

No. Jev's weights are not public. The Jev alternatives above are independent open models, mostly Apache 2.0, that answer the same kind of typed questions. Several of them also accept the same POST /v1/systemone requests.

Which Jev alternative runs on a CPU?

The encoder models are the lightest: Laya (322M and 421M parameters), Von (395M) and GLiNER2.5-Decide (340M). See Laya on CPU for measured latency.

Can I keep using the TypeSafe SDK?

Yes, with a server that speaks the Jev API. Kev and ollaya say the SDK works unchanged, and laya-serve says an existing Jev client only needs a new base URL.

System One models compared: all twelve models side by side
Laya vs Jev: open weights, speed, accuracy and tradeoffs
Self-host Laya: run a Jev-compatible API yourself
Laya on CPU: latency and sizing without a GPU

Last verified: September 25, 2026.

I maintain laya-ai.com, an independent Laya resource site; this article first appeared there as Jev alternatives. The site also keeps an auto-updated list of System One models on GitHub. It was drafted with AI assistance, and every fact was checked against the project pages linked above.

How fast is Laya on a CPU? Reading the 4-core server benchmarks

Yanng — Fri, 25 Sep 2026 08:56:57 +0000

Can you run Laya without a GPU? Yes, and the upstream project now publishes enough CPU measurements to say how fast, what it costs per question, and which settings matter. This guide reads those numbers so you can size a CPU deployment before you buy hardware.

All figures below come from the upstream Laya BENCHMARKS.md. Each is tied to the hardware and setup it was measured on.

The short answer

On a 4-core server CPU, laya-multilingual answers one question in about 193 ms; the English and typed-decisions checkpoints take about 580 ms.
On CPU, cost grows almost linearly with the number of questions. Batching questions into one call saves little, unlike on a GPU.
Thread settings can matter more than the model. On one laptop, fixing torch's thread defaults made calls about 12x faster.
Cold loads take seconds, and several loaded checkpoints can use around 9 GiB of RAM, so preload what you serve.

The server CPU numbers

Measured in-process on an AWS m7a.xlarge (AMD EPYC 9R14, 4 physical cores, no SMT, 16 GiB RAM), Laya v0.3.20, fp32, OMP_NUM_THREADS=4. Each call alternates a 3-option choice and a noul question. Values are p50; p95 was within 2% of p50 on every row.

checkpoint	1 question	5	10	50	cold load
`laya` (English)	580 ms	3,072 ms	6,244 ms	35,969 ms	4.4 s
`laya-multilingual`	193 ms	912 ms	1,842 ms	11,157 ms	2.5 s
`laya-typed-decisions`	584 ms	2,819 ms	6,031 ms	35,653 ms	0.5 s

Two things stand out.

The multilingual checkpoint is about 3x faster on CPU. It is built on mmBERT-base (322M parameters) rather than ModernBERT-large (421M). The upstream write-up does not break down where the rest of the gap comes from, so treat 3x as a measured result on this machine, not a rule.

The p95 is almost the same as the p50. On an otherwise idle server, CPU latency was very predictable. In our reading, that makes capacity planning simpler than on a shared GPU.

Why batching does not help on CPU

Up to 10 questions per call, each extra question costs about 600 ms on the English and typed-decisions checkpoints and about 185 ms on multilingual. At 50 questions the cost per question rises by another 15 to 20%. In other words, ten questions in one call take roughly as long as ten separate calls.

GPUs behave differently:

hardware	1 question	50 questions	cost per extra question
Tesla T4, `laya`	39.5 ms	771.3 ms	about 15 ms
NVIDIA GB10, `laya-typed-decisions` (over HTTP)	100.2 ms	443.1 ms	about 7 ms
EPYC 4-core CPU, `laya`	580 ms	35,969 ms	about 600 ms up to 10 questions, more beyond

On the GB10, roughly 93 ms of every call is fixed overhead, so packing questions into one call is where the speedup is. On CPU there is almost no fixed overhead to amortize, so the design choice is simpler: ask only the questions you need.

Thread settings: the easiest 12x

The upstream project also reports a laptop measurement (Ryzen 9 6900HX, WSL2) that is worth copying before anything else. With torch's default thread settings on a busy host (10 intra-op and 5 inter-op threads on 10 vCPUs), a three-question call over HTTP took 9,396 ms at p50. Setting two lines brought it to 783 ms, about 12x faster with no code change:

import torch

torch.set_num_threads(8)          # about the number of physical cores
torch.set_num_interop_threads(1)  # one forward pass per call: nothing to overlap

On the same laptop, one question in-process took 910 ms with 1 thread, 374 ms with 4, 329 ms with 8, and got worse again at 388 ms when every vCPU was used. The upstream advice is to use the physical core count plus a little, not one thread per vCPU, because SMT siblings contend. If you run the self-hosted server, LAYA_THREADS caps the same setting.

Cold starts and memory

A cold load took 4.4 s for the English checkpoint and 2.5 s for multilingual on the EPYC machine. These figures depend on the OS file cache, so treat them as approximate. Two practical consequences:

Preload the checkpoints you serve (Router(preload=True)) instead of loading on the first request.
Budget memory. The benchmark script peaked at 9.3 GiB with up to five checkpoints loaded at once. Serving fewer checkpoints should need less, but the upstream run does not report a per-checkpoint figure, so measure on your own machine.

When is a CPU enough?

This table is our interpretation of the numbers above, not a benchmark result. It assumes one question per call:

your situation	CPU is fine?
Background jobs, queues, nightly batch classification	Yes. Latency rarely matters; cost does.
Interactive routing or triage in non-English text	Usually. `laya-multilingual` at about 0.2 s is fast enough for many UIs.
Interactive English decisions with several questions per request	Borderline. Five questions on the English checkpoint take about 3 s.
High-throughput, low-latency serving	Use a GPU. A T4 answers one question in 32.8 to 39.5 ms.

These are single-process, in-process numbers. Real deployments add HTTP, queuing, and concurrency effects, so benchmark your own hardware with your own questions before committing. The upstream script is research/scripts/bench_latency.py.

Sources

Laya BENCHMARKS.md: server CPU, laptop CPU, T4, and GB10 sections
Raw EPYC results
Laya benchmarks tracker: upstream and independent results with their test conditions