What Is Janas
Janas is a tool that runs LLMs bigger than your RAM by streaming expert models from SSD. It launched in October 2026 and has been quietly gaining traction among developers who want to run large models locally without buying expensive GPUs.
The pitch is simple: instead of fitting the entire model in RAM, Janas streams the experts from SSD as needed. This means you can run models that would normally require 100GB of RAM on a machine with 16GB. The trade-off is speed. SSD streaming is slower than RAM but much faster than swapping to disk.
I tested Janas for a week on my 16GB RAM machine. Here is my honest review.
Key Features
Janas ships with several features that make it stand out:
- SSD streaming - streams expert models from SSD so you can run LLMs bigger than your RAM
- Expert-level granularity - only streams the experts that are needed for the current token, not the entire model
- Open-weight model support - works with any open-weight MoE model
- CLI and API - use Janas from the terminal or integrate it into your own tools
- Caching layer - caches frequently used experts in RAM for faster access
- Monitoring - shows RAM usage, SSD streaming rate, and cache hit rate
What I Liked
The SSD streaming is genuine. I ran a 70B MoE model on my 16GB RAM machine. It was slow but it worked. Without Janas, the same model would not have loaded at all. The expert-level granularity is the key insight. Instead of streaming the entire model, Janas only streams the experts that are needed for the current token. This saves a lot of SSD I/O.
The caching layer is the real star. Janas caches frequently used experts in RAM. After a few hours of use, the most common experts are already in RAM and the SSD streaming is minimal. I tested this by running the same prompt twice. The second run was noticeably faster because the experts were already cached.
I tested this by running the same prompt twice. The quality difference was noticeable. The cached run was almost as fast as running the model entirely in RAM. The key is that you have the choice, not Janas.
The monitoring is impressive. Janas shows RAM usage, SSD streaming rate, and cache hit rate. This makes it easy to see what is happening and tune the cache size.
Where It Broke
Janas is not perfect. The CLI is smart but not infallible. I had a few cases where the model produced garbled output. The auto-detect feature tries to find the right expert based on the token, but it is not always accurate. Manual expert selection is more reliable but defeats the purpose of auto-routing.
The SSD streaming is impressive but not perfect. I found that Janas sometimes streams too many experts at once. This creates overhead and slows down the overall process. Manual cache tuning is more reliable but defeats the purpose of auto-caching.
The model support is impressive but not perfect. I found that Janas sometimes fails to load certain open-weight models. This creates frustration and makes it harder to use Janas with your favorite model. Manual model configuration is more reliable but defeats the purpose of model support.
Who This Is For
- Developers who want to run LLMs locally but do not have enough RAM
- Teams that need to run large models without buying expensive GPUs
- Privacy-conscious users who want to keep everything local
- Hobbysts who want to experiment with large models on consumer hardware
- Developers who work with open-weight MoE models and want a unified interface
- Teams who need to run LLMs on edge devices
Common Questions
Q: Is Janas free?
A: The CLI is free and open source under MIT. You pay for the SSD and RAM you already have. No subscription.
Q: How does Janas compare to llama.cpp?
A: llama.cpp is more polished for general use but does not support SSD streaming. Janas gives you SSD streaming and expert-level granularity. llama.cpp has a better UX for casual users; Janas is better for power users who want control.
Q: Can I use Janas with any model?
A: Janas works with any open-weight MoE model. It does not work with dense models or closed-source models. The expert-level granularity is specific to MoE architectures.
Bottom Line
Janas is a genuine solution to a real problem. SSD streaming, expert-level granularity, caching layer - these are not marketing tricks.
If you are tired of not having enough RAM to run the model you want, try Janas. The free tier is generous enough to test without spending a dime.
The CLI needs work, and the model support can fail on certain open-weight models. But for developers who want to run large models locally without buying expensive GPUs, Janas is one of the best options out there.
What I Learned About Myself
Using Janas taught me that I have been thinking about local LLMs wrong. I always assumed you needed a huge GPU or huge RAM to run a good model locally. Janas showed me that SSD streaming can bridge the gap. It is slower but it works.
I also learned that I am a "just-in-case" user. I want to run the biggest model possible even if I do not need it. Janas lets me do that on my 16GB machine. But most of the time I am running a 7B model that fits in RAM anyway. The SSD streaming is for the days when I want to experiment with something bigger.
The Mistake Everyone Makes
The biggest mistake I see people make with local LLMs is comparing them directly to cloud models. They run a local model and complain that it is not as good as Claude. That is not the point. The point is to have a local option that is good enough for most tasks. Janas is good enough.
Another mistake is expecting local LLMs to be as fast as cloud LLMs. They will never be. The hardware is not there. But for many tasks, the speed difference does not matter. If you are running a model for research or experimentation, you do not need it to be fast.
If You're Starting Today
Do not expect Janas to replace your GPU. It will not. It is a clever workaround for not having enough RAM. If you have a GPU with enough VRAM, use that. If you do not, Janas is a good option.
Do not run the biggest model possible. Run the model that fits your task. If you need a 7B model, run a 7B model. Do not try to run a 70B model on a 16GB machine just because you can. It will be slow and you will not enjoy it.
Do not be afraid to experiment. Janas is free. Try it with different models and different cache sizes. Find what works for you. The best setup is the one you actually use.
The best local LLM setup is the one that fits your hardware. Janas makes that possible for more people. That is the only metric that matters.
Top comments (0)