Someone on GitHub asked if jevmem could run fully local.
Fair question. Today jevmem needs a paid TypeSafe API key: it sends each message, with common secrets removed, to TypeSafe's Jev API to decide what's worth saving.
Ollaya runs open decision models on your own machine, and it answers in the same format as Jev's API. So I pointed jevmem at it.
It connected with no code change: one URL, any key, and the model's name in the config.
Then I ran the same tests I use for Jev.
Setup
2 Oct 2026. M4 Mac with 16 GB, Ollaya 0.9.0. One run each, next to a Jev run the same afternoon.
Deciding what to save (66 held-out turns)
| Save/skip right | Save + kind right | Median per turn | |
|---|---|---|---|
| Jev | 65/66 | 63/66 | 0.23 s |
| winnow:e4b (8 GB) | 60/66 | 57/66 | 28.5 s |
| laya:typed-decisions (852 MB) | 19/66 | 19/66 | 13.6 s |
Both local models needed the client timeout raised from 10 s to 180 s.
Recall and the guard (laya only)
- Recall: no answer came back within jevmem's 1-second budget, so every prompt fell back to word matching: 48 of 78 wanted lines, against Jev's 73. Its 1,024-token context can't hold an 18-line memory file anyway.
- Guard: fast enough (158 ms median), but it caught 43 of 68 rule breaks with 13 false asks. Jev caught 66 with 3.
What it means
On a laptop, not yet, so jevmem 0.7.3 still uses Jev. The accurate model is over 100× slower. The fast one is wrong too often.
But 60 of 66 is close. And on JevBench, larger open models score near Jev when they run on servers. So this may change with better hardware, or the next models.
If you have an NVIDIA GPU and want to try, scripts/local-model-ollaya.sh in the repo runs each test against a local server. I'd love to see your numbers.
Repo: https://github.com/Avinash-jetwani/jevmem · Results: https://avinash-jetwani.github.io/jevmem/results/#can-a-local-model-do-jevs-job · JevBench: benchmarkheaven.com/jev-models
Written with help from Claude. The numbers are from my own test sets, run once each, and the results files are in the repo.
Top comments (0)