OpenJev runs a local language model in your browser and does one specific thing with it. Given a prompt and a fixed set of options, it reads the model's logits for those option tokens and applies a softmax across only them. There is no decoding and no generated text; a single forward pass yields a probability distribution over the answers you allowed. The site calls these "semantic ifs": a branch decided by the model's weighting of your options instead of a hand-written condition.
The demo exists to measure that readout against the obvious alternative. The second method hands the same prompt and the same options to the same model and asks it to write the distribution as JSON, options and probabilities, token by token. Both run sequentially on one loaded model so they do not contend for the GPU; the direct readout runs first, then generation. Setup, warmup, prompt preparation, direct execution, first generated token and completion are all timed with performance.now(), and the page shows nothing canned.
The result an engineer cares about is the timing gap. Reading logits over your options is a single forward pass. Generating JSON is autoregressive: you pay first-token latency and then every token of the output. For anything that is genuinely a constrained choice (routing, classification, a yes/no gate), the model does not need to produce prose that you then parse. The distribution is already in the logits, and reading the right tokens off it is cheaper than making the model spell it out.
OpenJev is clear about what the direct numbers are. They are a softmax over only the displayed option tokens, which makes them conditional probabilities rather than calibrated confidence, and they exclude any answer the model might have preferred outside your list. The option labels shown run A through T, twenty slots. Weights come from pinned GGUF builds served through wllama; quantization can shift both quality and speed, and the browser build may not match the native checkpoint. MiniCPM5 2B is the default, Qwen3 0.6B is there for phones, and the 4B option needs substantially more memory. The model is about 1.56 GB, it loads from Hugging Face into the browser cache, and inputs never leave the page.
The quality table is less forthcoming. It scores the local models on balanced accuracy against two reference columns. One is "TypeSafe," defined as equal-case agreement on a 102-row public subset. The other is "Jev," listed only as "the published value," and the caveats add that none of the local tiers is claimed to match it. TypeSafe comes with a definition; Jev gets none. The page never says what that published value measures or where it comes from.
That matters if you are deciding whether to lean on this. The readout method is clear and testable on your own hardware, which is the entire point of shipping it as a live demo; you can watch the two latencies for yourself. The accuracy numbers are harder to check. Measured quality, the page says, comes from native checkpoints against owned and public benchmarks, but one of those reference columns is never more than "the published value." Run the timing comparison and trust what your GPU reports, because the quantized browser weights can diverge from the native checkpoint. Treat the Jev column with caution until the page says what that published value actually is.
Top comments (0)