Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s

The wall everyone hits with local models is always the same one: the model you actually want to run is bigger than your memory. You find a promising 80B mixture-of-experts model, check the file size, see 48 GB, look at your 32 GB of RAM, and close the tab. Or worse, you try it anyway and watch your machine drown in page faults while tokens crawl out at 3 per second.

A project that hit the Hacker News front page this week claims to break that wall. It is called Janas, a language model engine written in C by Maurizio Cammalleri, and its headline number is striking: 37 tokens per second from Qwen3-Next-80B, a 48.4 GB model, on a laptop with 32 GB of RAM. The same model on llama.cpp on the same laptop: 2.8 tokens per second.

Full disclosure before anything else: I have not run Janas myself. Every number in this article comes from the project's own published benchmarks, its README, and the Hacker News discussion, all linked inline. Where a claim is the author's own measurement rather than an independent one, I say so. Treat this as a guided tour with a checklist, not a personal success story.

Why models bigger than RAM usually crawl

The obvious question is: does not the operating system already handle this? If the model file sits on an SSD, the OS can memory-map it and page in pieces as needed. Tools like llama.cpp do exactly that with mmap.

The problem is what happens when the working set exceeds memory. A mapped file does not know which parts of the model matter. Every page fault is a trip through the kernel's page cache machinery, and when memory runs out, the kernel starts evicting pages you are about to need. Researchers who studied this problem describe it as a fault storm: at around twice your RAM in model size, the mmap approach collapses, because the OS is juggling pages blindly while your inference stalls waiting on disk.

Mixture-of-experts models make this worse in a specific way and also hint at the fix. A MoE model contains many separate "expert" feed-forward blocks, and each generated token only routes through a few of them, often 8 out of 256. Most of those billions of parameters sit idle for any given token. The bottleneck is not reading the model, it is reading the right part of the model, fast, every single token.

What Janas does differently

Janas attacks that problem directly, and the design is easier to trust because it is simple to state:

• Weights stay on disk. The engine never loads the whole model. It keeps a converted .jns file on your NVMe SSD and reads expert weights with O_DIRECT, bypassing the page cache entirely so the OS never starts thrashing.
• Hot experts get cached. As the model runs, the engine notices which experts keep coming back and holds those in a memory cache. It measures your machine at startup and sizes its own caches and thread choices from what it finds.
• The GPU reads prompts, not the whole model. On machines with an integrated GPU, Janas uses it for prompt processing, where parallelism helps most, and streams expert weights for token generation. You can force it CPU-only with JANAS_GPU=0.
• Draft tokens hide latency. The engine supports speculative drafting: a small block guesses the next few tokens and the big model checks them in one pass. On Qwen3-Next-80B that took writing speed from 24 to 37 tok/s, because the text is identical, just produced faster.

The result on the author's test laptop, a Debian 13 machine with an Intel Core Ultra 9 185H, 32 GB of DDR5, and a 1 TB NVMe SSD reading at about 5.7 GB/s: Qwen3-Next-80B reads a 1,024-token prompt at 62 tok/s and writes at 24 tok/s without drafts. Qwen3-Coder-Next, also 48.4 GB, does 66 and 22.

The benchmark numbers, model by model

The author reports Janas beat llama.cpp on all 13 models compared, on the same laptop, in Q4_K_M quantization, measured over 9-11 October 2026. These are the author's own measurements, so weigh them accordingly. The headline rows:

• Qwen3-Next-80B-A3B (48.4 GB, larger than the RAM): 62 tok/s reading vs 16, 24 tok/s writing vs 2.8. That is roughly 8.5x faster at writing.
• Qwen3-Coder-Next (48.4 GB, larger than the RAM): 66 tok/s reading vs 14, 22 tok/s writing vs 2.2.
• Qwen3.6-35B-A3B (22.3 GB): 104 vs 79 reading, 19 vs 12 writing.
• Qwen3-30B-A3B (18.6 GB): 135 vs 122 reading, 32 vs 15 writing.
• Gemma-4-26B-A4B-it (18.3 GB): 90 vs 76 reading, 18 vs 12 writing.
• Small dense models: Qwen3-0.6B reads at 939 tok/s vs 670. Even here, Janas is faster.

One nuance worth knowing: for the oversized model, llama.cpp did not simply fall behind, it broke. With its usual build it ran out of memory as soon as the GPU joined, because its CPU backend copies weights to repack them and its Vulkan backend copies what it leaves on the CPU into buffers the system cannot reclaim. The 2.8 tok/s figure required a special build without repacking plus flags that leave the experts in the file. Janas ran the same model with no special options.

Skeptics in the Hacker News thread made the fair point that llama.cpp has had mmap-based disk streaming for years, and other projects like Strata and flash-moe work the same territory. The difference in the numbers, if they hold up on other machines, is that Janas measures your hardware and actively manages which experts live in memory, rather than leaving page-in decisions to the kernel. Whether that gap survives independent testing is exactly the kind of thing the janas-bench report is meant to answer, and the author explicitly says results on untested hardware are unknown.

How to try it yourself

If you have a Linux x86-64 machine (Ubuntu 22.04, Debian 12 or later) and want to reproduce or falsify these numbers, the quickest path from the README:

curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz
tar xf janas-linux-x86_64.tar.gz && cd janas
./janas-chat

Pick a model from the catalog and it downloads and verifies it. On Windows 10 (1903+) and 11 there is a zip release; the author reports writing speed on Windows 11 matched Linux, and dense models read prompts 1.8x faster there because whole blocks go to the GPU through Intel's own driver.

Practical flags worth knowing before you start:

• --cache <GiB> puts a hard ceiling on the memory Janas takes. The engine fills free memory with its expert cache by default, which is polite to nobody.
• JANAS_GPU=0 keeps it off the GPU entirely. The README honestly notes the GPU path once caused a driver reset that took the desktop down with it.
• janas-bench measures your machine and produces a report. If you run it anywhere, that report is the single most useful thing you can send the project, because it has so far been tested on exactly one laptop.
• Close your browser first. With a browser and editor open, the same model kept about three quarters of its speed but waited five times longer on disk. Run your first test on an idle machine.

The catches, stated plainly

• It is an experiment, not a product. The command line has changed several times a week. Only the .jns model file format is promised stable.
• One machine, one author. Everything was developed and measured on a single Intel laptop with an integrated Arc GPU. AMD CPUs, discrete GPUs, and slower disks are explicitly untested territory.
• It is deliberately hard on your machine. Full-memory caches plus full-speed O_DIRECT reads can push a machine without headroom into swap. The author says it degrades gently, but budget your cache anyway.
• Model availability is the real constraint. You need converted .jns files, which currently means the author's Hugging Face catalog of Qwen and Gemma models, or converting GGUF files yourself. Your favorite model may not exist in this format yet.
• Draft speedup depends on the text. Drafts survived 74-89% of the time on English prose in the author's tests, but on Italian prose one model kept only about a third. The README is candid that an earlier version of that benchmark was flattering and wrong, and it was corrected.

Should you use it? A decision checklist

Here is how I would decide, given that these are published numbers rather than my own testing:

• Your model fits in RAM: stay on llama.cpp. It is mature, universal, and the gap on small models, while real in the benchmarks, buys you little.
• Your model is bigger than your RAM and you have a fast NVMe SSD: Janas is exactly built for this case, and 24-37 tok/s on an 80B model changes it from a curiosity into something usable. Verify with janas-bench before you believe it on your hardware.
• You need reliability or a specific model: the ecosystem and tooling around llama.cpp are years deep. Janas lists no stability guarantee at all.
• You just need good answers and have a connection: a paid API still beats every local setup on quality per token of effort. Local models are about privacy, cost at scale, and offline use, not beating frontier APIs.
• You enjoy the edge: run janas-bench and send the report. Projects like this live or die on measurements from machines the author does not own.

The bigger story here is that the definition of "too big to run locally" is now tied to your SSD bandwidth, not your RAM capacity. An NVMe drive moving 5-7 GB/s can stream more expert weights per second than a token generator consumes. If that gap holds, the practical ceiling on local models stops being your memory and starts being your disk, and disks are cheaper to upgrade. The author has written more about the engine on his Substack, and the Hacker News discussion is worth reading for the skeptics' counterpoints.

I write about AI infrastructure, developer tools, and backend engineering every week. Subscribe, it is free.

Have you pushed a local model past your RAM limit? What happened when you tried, and did anything actually work? I am collecting real-world results on this one, so tell me in the comments.