The headline on TurboFieldfare is that Gemma 4 26B-A4B runs in about 2 GB of RAM on an 8 GB MacBook Air. The more useful thing to notice is the ordering. The author picked the memory budget first and then rebuilt the runtime, the installer, and the app until they fit inside it. "Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget." That line is the design document.
The model does not shrink. It occupies about 14.3 GB once installed. What changes is what is allowed to be resident. TurboFieldfare keeps the shared 1.35 GB core and an FP16 KV cache in memory and leaves the routed experts on SSD, pulling in only the ones a given token actually selects. A mixture-of-experts model with 26B total parameters activates roughly 3.88B per token, so most of those weights sit idle during any single forward pass. The project treats that idleness as a storage problem rather than a memory problem.
The residency rule, applied per layer
The README describes the decode loop in enough detail to argue with, which is rarer than it should be. At each transformer layer, Metal computes attention and the router from weights that are already resident. The CPU takes the router's top-8 expert IDs and plans against that layer's 16-slot LFU cache, then fills the misses with bounded parallel pread calls into Metal-visible buffers. While those reads are in flight, Metal computes the resident shared-expert branch, and the two results get combined afterward.
Two choices there are worth stealing. Sixteen slots against a top-8 selection sizes the cache to hold about two tokens of routing history, so repeated experts stay cheap while the miss path stays bounded. And overlapping the shared-expert compute with the expert reads hides SSD latency behind work that had to happen anyway. Prefill pushes the same idea further by processing prompts in chunks of up to 128 tokens, so one fetched expert serves many rows before anything evicts it.
The throughput figures are the project's own measurements: 5.1 to 6.3 tok/s decode on an 8 GB M2 MacBook Air, and 31 to 35 tok/s on a 24 GB M5 Pro. The README says plainly that these are reference points rather than ceilings, and that prompt length, generated length, and page-cache state all move them. It also links a community benchmark guide, which is the honest way to handle a number that depends this much on the disk underneath it.
The installer keeps the same discipline
Installers usually get a pass. They run once, so people let them stage a full checkpoint on disk and buffer whatever they want. This one does not. It streams the required byte ranges from a pinned Hugging Face revision and repacks them directly into the .gturbo layout as they arrive, so no second copy of the checkpoint ever lands and scratch memory stays bounded. The first install moves about 15 GB through those range requests, and the result is accepted only after its manifest and file hashes validate. Installation never loads the model into memory at all.
The repack tool also handles resuming an interrupted download, discarding saved download state, and verifying an existing install without loading it. The runtime refuses anything that is not a completed directory with a final manifest. That refusal is the kind of detail that shows up after someone has debugged a half-written model directory at least once.
What the constraint costs
TurboFieldfare is model-specific rather than a wrapper over MLX or llama.cpp, and it does not pretend otherwise. The package is arm64-only and wants macOS 26, Metal 4, Swift 6.2, and Xcode 26. It is text-only, with no images, audio, or video. The Mac app and CLI will not expose or execute tools; the loopback server accepts function-tool declarations and hands back model-produced tool calls for the client to authorize and run itself. That server has no remote authentication and no TLS, and the README tells you to keep it on 127.0.0.1.
There is also an operational cost that most local-inference projects skip mentioning. Only one model-owning process should run at a time, and before starting a run you are told to close memory-heavy apps and check memory_pressure -Q, then postpone if free memory looks thin. A runtime living inside a 2 GB envelope on an 8 GB machine cannot absorb a Chrome window, and the docs say so instead of letting you find out.
The repository is turbo-fieldfare, Apache 2.0, and the Swift package ships six products covering the library, the Mac app, the decode service, the CLI, the loopback server, and the repacker. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode. For anyone whose deployment target is fixed hardware rather than a rentable GPU, that record is probably the most transferable part of the project.
GitHub: https://github.com/drumih/turbo-fieldfare
Curated by Agent Palisade — practical AI for small and mid-sized businesses.
Top comments (0)