AI OpenFree

Posted on Nov 1

Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads

#ai #machinelearning #llm #edgeai

TL;DR

VIDRAFT POCKET is a family of GGUF models that run large language models on hardware people already own: a PC with no GPU, a laptop, and (with the phone builds) an iPhone or Android device. It works with stock llama.cpp, Ollama, LM Studio and PocketPal. No fork, no CUDA, no cloud.

As measured on the Hugging Face public API on 1 November 2026, the three flagship POCKET repositories were downloaded 1,113,164 times in the last 30 days and 3,154,913 times in total.

Model	Last 30 days	All-time
POCKET-35B-GGUF	526,862	1,948,924
POCKET-Darwin-180B-GGUF	343,743	343,743
POCKET-26B-GGUF	242,559	862,246
Total	1,113,164	3,154,913

POCKET-Darwin-180B was published less than 30 days ago, so its 30-day and all-time counts are the same.

What is VIDRAFT POCKET?

POCKET is our answer to a simple question: how large a model can a normal person run, privately, on the device in front of them?

Most "small" models still assume a datacenter GPU somewhere. POCKET takes the opposite starting point. Each repository ships ready-to-run GGUF files, quantized so that the model fits in the RAM of a consumer machine, and tested on the tools developers already use.

Three properties define the family:

Stock runtimes only. Every file loads in upstream llama.cpp and the apps built on it. We do not ask you to install a custom fork.
Sparse Mixture-of-Experts models. Only a small part of the network is active for each token, which is why a 35B or 180B model can generate at usable speed on a CPU.
Measured, not estimated. Every speed and quality number on the model cards comes from a run we did ourselves, on named hardware. Where we have not measured something (for example iPhone throughput for every build), the card says so.

Which POCKET model should I download?

Repo	Typical file	Size	Runs on	Best for
POCKET-35B-GGUF	`Q2_K`	13 GB	mini-PC or desktop, no GPU	daily driver
POCKET-35B-GGUF	`Q4_K_M`	21 GB	PC with 32 GB RAM	top quality
POCKET-35B-GGUF	`IQ1_M`	8.2 GB	16 GB RAM machine	smallest full model
POCKET-26B-GGUF	`Q2_K`	11 GB	any app today (Ollama, LM Studio, PocketPal)	compatibility, Korean
POCKET-Darwin-180B-GGUF	4-bit GGUF	111 GB	laptop with 8 GB VRAM and 32 GB RAM, or a 128 GB RAM mini-PC	flagship quality on a desk
POCKET-KR-MLX / POCKET-EN-GGUF	phone builds	about 5 GB	iPhone, iPad, Android	on-phone use

A quick rule: start with POCKET-35B Q2_K on a PC, POCKET-26B if your app does not yet support the newest architectures, and POCKET-Darwin-180B if you want the strongest model and have the disk space.

How fast is a 35B model on a CPU?

We compare against Bonsai-27B because it is the closest model in size that targets the same "large model, small machine" niche. Both run in the same stock llama.cpp build on the same machine.

On a MacBook M3 Pro with 18 GB (measured):

	POCKET-35B IQ1_M	Bonsai-27B Q1_0	Ratio
CPU generate, 8 threads	13.8 tok/s	4.4	3.13x
CPU prompt, pp128	45.5 tok/s	9.6	4.75x
Metal generate, tg64	25.4 tok/s	12.8	1.99x
Metal prompt, pp128	240.7 tok/s	73.4	3.28x

On a 16-thread Xeon CPU, POCKET-35B IQ1_M generates at 27.0 tok/s against 10.1 for Bonsai-27B (2.69x). On an H100, generation is 197 against 89 tok/s. We also publish where we lose: on the H100, Bonsai is faster at prompt processing (1816 against 753 tok/s). On HellaSwag (400 questions) the two models tie within the confidence interval (61.0% against 60.0%).

The reason POCKET wins generation is the Mixture-of-Experts design. A dense 27B model has to read every weight for every token. A sparse 35B model reads only the active experts, so memory bandwidth, the real limit on a CPU, goes much further.

What does quantization cost in quality?

We measure reasoning quality on GPQA Diamond (198 questions, greedy decoding):

Model	GPQA Diamond
POCKET-35B Q4_K_M	68.7%
POCKET-35B Q2_K	60.1%
POCKET-26B base (Gemma-4-26B-A4B)	67.7%
POCKET-26B Q2_K	67.2%

POCKET-26B keeps almost all of its base model's GPQA score at 2-bit because our mixed-precision quantization keeps the most quality-critical weights at higher precision. For POCKET-Darwin-180B, the 4-bit build matches the BF16 model on our quality check (87.65%).

How do I run POCKET with llama.cpp?

Any recent llama.cpp from Homebrew, winget or apt works. No fork is needed.

# download one file from Hugging Face
huggingface-cli download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-Q2_K.gguf --local-dir .

# chat on the CPU only (-ngl 0), use your physical core count for -t
llama-cli -m POCKET-35B-Q2_K.gguf -p "Explain mixture-of-experts in two sentences." -ngl 0 -t 8

# reproduce our CPU speed numbers
llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16

Two practical notes from our own runs:

Set -t to the number of physical cores, not logical threads. Passing every hardware thread can slow generation down sharply.
On an 18 GB Mac, run Q2_K on the CPU (-ngl 0). We measured 19.5 tok/s for POCKET-35B Q2_K on the M3 Pro CPU.

If you prefer an app, the same GGUF files load in Ollama, LM Studio and PocketPal.

Why are developers downloading POCKET?

We read the download curve as a signal about what people want from local AI:

Privacy by default. The model runs where the data is. Nothing leaves the device.
Offline by design. A laptop on a plane, a factory floor, an air-gapped office: none of them need a network.
No new hardware. The target is the machine you already have, not a GPU you have to buy.

The ecosystem around the family also grows on its own. Independent quantizers in the community publish their own conversions of our Darwin models, and the POCKET collection keeps adding builds for new devices.

How did we measure the download numbers?

All counts come from the Hugging Face public API (/api/models with expand=downloads for the last 30 days and downloadsAllTime for the total), read on 1 November 2026. Anyone can repeat the query. The full measurement page, including the method and a same-method comparison of models published in 2026, is here: vidraft.net/hf-usage.html.

FAQ

Can I run a 35B model without a GPU?
Yes. POCKET-35B Q2_K (13 GB) runs on a CPU-only PC with stock llama.cpp. On a MacBook M3 Pro CPU we measured 19.5 tok/s.

Does POCKET need a custom llama.cpp fork?
No. Every POCKET file loads in upstream llama.cpp, Ollama, LM Studio and PocketPal.

Can a 180B model really run on a laptop?
POCKET-Darwin-180B (4-bit, 111 GB) runs on a laptop with 8 GB of VRAM and 32 GB of RAM at a measured 4.17 tok/s, streaming weights from the SSD. A 128 GB RAM mini-PC runs it without a GPU.

Which POCKET model is best for Korean?
POCKET-26B is tuned for on-device Korean, and POCKET-KR-MLX / POCKET-KR-GGUF are the Korean phone builds.

How many times has POCKET been downloaded?
1,113,164 times in the last 30 days and 3,154,913 times in total across the three flagship repositories (Hugging Face, 1 November 2026).

Where can I try it without installing anything?
The POCKET collection lists every build, and POCKET Image Studio runs in the browser.

DEV Community