Run Gemma 4 26B on an 8GB Mac with TurboFieldfare

Run Gemma 4 26B on an 8GB Mac with TurboFieldfare

TurboFieldfare can run the text-only Gemma 4 26B A4B model on an 8GB Apple Silicon Mac with roughly a 1.9–2.1GB process footprint. It achieves that result by keeping a 4K FP16 KV cache and shared core resident while streaming routed experts from a 14.3GB SSD installation. This guide sets up the runtime and measures the configuration honestly.

The commands and API contract below come from the TurboFieldfare project documentation. This draft was not executed on Apple Silicon in the current authoring environment. Run the validation steps on the target Mac and add the real output before publishing.

What the 2GB number means

Gemma 4 26B A4B has approximately 25.2B total parameters and approximately 3.8B active parameters per token. Because each token uses eight of 128 routed experts plus a shared expert, TurboFieldfare can keep a bounded expert cache and stream missing experts from SSD.

The reference configuration is:

Item	Requirement or reported value
Hardware	Apple Silicon Mac; 8GB M2 MacBook Air is the validated entry point
OS	macOS 26+ with Metal 4
Toolchain	Xcode 26 and Swift 6.2+
Process footprint	Approximately 1.9–2.1GB in published runs
Installed model	Approximately 14.3GB
Initial transfer	Approximately 15GB
Headline KV context	4K FP16
Local modalities	Text only

The project reports 5.10–6.30 decode tok/s on an 8GB M2 MacBook Air and 31–35 tok/s on a 24GB M5 Pro. These are project measurements, not Google benchmarks, and they exclude installation, model loading, and prompt prefill.

Step 1: Check the Mac

Confirm the architecture, macOS version, Swift toolchain, free storage, and memory pressure:

uname -m
sw_vers
swift --version
df -h .
memory_pressure -Q

Expected prerequisites:

• uname -m returns arm64.
• macOS is version 26 or later.
• Swift is 6.2 or later.
• At least 16GB of free storage is available for the initial installation; leave additional headroom for builds and normal system operation.
• Memory pressure is healthy before loading the model.

Close other local-model runtimes and memory-heavy applications. TurboFieldfare’s documentation recommends running one model-owning process at a time.

Step 2: Build TurboFieldfare

Clone the repository and build the release package:

git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release

Launch the native Mac app:

.build/release/TurboFieldfareMac

In the app:

1. Choose Download.
2. Wait for the roughly 15GB transfer and repack.
3. Confirm that the completed model occupies about 14.3GB.
4. Choose Load Model.
5. Start with the default 4K context and 16 expert-cache slots.
6. Enter a short prompt and generate a response.

The installer streams remote byte ranges into the .gturbo layout and validates the completed manifest and file hashes. It does not need to materialize a second full checkpoint.

Step 3: Verify an installation from the CLI

If you prefer the command line, install or resume the model with:

swift run -c release TurboFieldfareRepack \
  --output scratch/gemma4.gturbo \
  --overwrite \
  --resume

Verify the completed installation without loading the model:

swift run -c release TurboFieldfareRepack \
  --verify-install \
  --input-gturbo scratch/gemma4.gturbo

Do not treat a partial directory as a model installation. The runtime accepts a completed .gturbo directory with a final manifest.json.

Step 4: Run a deterministic CLI smoke test

Use a short raw completion with greedy decoding:

swift run -c release TurboFieldfareCLI \
  --model scratch/gemma4.gturbo \
  --prompt "The capital of France is" \
  --max-new 64 \
  --temperature 0

Generated text goes to standard output, while timing statistics go to standard error. Record the commit, hardware model, OS, Swift version, prompt length, generated tokens, context setting, expert-cache slots, TTFT, decode rate, and peak memory.

One short completion is a smoke test, not a benchmark.

Step 5: Start the local Chat Completions server

First check that another model-owning process is not running:

pgrep -fl 'TurboFieldfareServer|TurboFieldfareMac|TurboFieldfareDecodeService|TurboFieldfareCLI|mlx_lm|mlx-lm'

If the command prints a live model process, stop it normally before starting the server.

Build and start the loopback server:

swift build -c release --product TurboFieldfareServer
.build/release/TurboFieldfareServer \
  --model scratch/gemma4.gturbo \
  --port 8080 \
  --max-context 4096

Using --max-context 4096 keeps this tutorial aligned with the 4K context behind the approximately 2GB headline. The project’s documented server example uses 16K, but larger FP16 KV caches consume more memory.

Wait for TurboFieldfareServer ready, then use another terminal for the client checks.

Step 6: Check health and the model ID

curl --silent --show-error http://127.0.0.1:8080/health
curl --silent --show-error http://127.0.0.1:8080/v1/models

Send a deterministic request:

curl --silent --show-error http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma-4-26b-a4b-it",
    "messages": [
      {"role": "user", "content": "Reply with exactly READY."}
    ],
    "temperature": 0,
    "max_completion_tokens": 16
  }'

The local model ID matches the official hosted model ID, but the interfaces are not feature-equivalent. TurboFieldfare’s server supports Chat Completions, streaming, function-tool declarations, and single-prefix reuse. It does not support the Responses API, embeddings, multimodal input, structured output, batching, log probabilities, or remote model switching.

Step 7: Measure TTFT and decode throughput

Create a separate Python environment and install the OpenAI client:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip 'openai>=1.60,<3'

Save this as measure_local_gemma.py:

from __future__ import annotations

import json
import time
from dataclasses import asdict, dataclass

from openai import OpenAI


@dataclass
class Result:
    time_to_first_text_seconds: float | None
    total_seconds: float
    completion_tokens: int | None
    measured_decode_tokens_per_second: float | None
    text: str


client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key="local",
)

started = time.perf_counter()
first_text_at: float | None = None
parts: list[str] = []
completion_tokens: int | None = None

stream = client.chat.completions.create(
    model="gemma-4-26b-a4b-it",
    messages=[
        {
            "role": "user",
            "content": (
                "Explain mixture-of-experts routing in 150 to 200 words. "
                "Use plain language and no bullet points."
            ),
        }
    ],
    temperature=0,
    max_completion_tokens=256,
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    now = time.perf_counter()
    if chunk.choices:
        content = chunk.choices[0].delta.content or ""
        if content:
            if first_text_at is None:
                first_text_at = now
            parts.append(content)

    if chunk.usage is not None:
        completion_tokens = chunk.usage.completion_tokens

finished = time.perf_counter()
text = "".join(parts)
ttft = None if first_text_at is None else first_text_at - started
decode_seconds = None if first_text_at is None else finished - first_text_at
decode_rate = (
    None
    if not completion_tokens or not decode_seconds or decode_seconds <= 0
    else completion_tokens / decode_seconds
)

result = Result(
    time_to_first_text_seconds=ttft,
    total_seconds=finished - started,
    completion_tokens=completion_tokens,
    measured_decode_tokens_per_second=decode_rate,
    text=text,
)

print(json.dumps(asdict(result), indent=2, ensure_ascii=False))

Run it while the local server is active:

python measure_local_gemma.py | tee local-gemma-result.json

The client-side decode calculation is approximate because the final usage chunk and network buffering affect the clock. Use the server’s timing output as the primary decode figure and the script as an end-to-end client observation.

Step 8: Create a reproducible result table

Do not publish one run as a hardware guarantee. Run a warmup, then repeat fixed prompts in fresh processes and record every condition.

Field	Result
Git commit	[commit]
Hardware	[Mac model / RAM]
macOS / Swift	[versions]
Prompt tokens	[value]
Generated tokens	[value]
Context / expert cache	4K / 16 slots
File-cache state	[cold / warm / uncontrolled]
TTFT	[seconds]
Server decode rate	[tok/s]
Peak RSS / footprint	[MiB]
Output accepted?	[yes/no + rule]

The project’s own benchmark protocol uses fixed prompts and requires coherent, non-repeating output. Follow that methodology before describing a result as comparable.

Keep the server on loopback

The local server binds to 127.0.0.1 and has no authentication or TLS. The project explicitly says not to expose it through a proxy or tunnel.

For local tools, keep the client on the same Mac. For remote or multi-user production access, do not simply change the bind address. Design a separate serving layer with authentication, authorization, TLS, request limits, queueing, process supervision, readiness checks, memory-pressure and SSD metrics, privacy-aware logging, overload handling, and failover.

Model-generated tool calls also require client-side approval. The server returns a proposal; the application decides whether the tool may execute.

Local versus official API

Google’s official hosted model ID is also gemma-4-26b-a4b-it. The Gemini API provides text and image input, managed infrastructure, function calling, and model context up to 256K.

Google currently lists Gemma 4 on its API free tier and does not list a paid Gemma 4 tier. Recheck the pricing page and data-use terms before production use. Free-tier content may be used to improve Google products.

As of July 30, 2026, CometAPI does not list Gemma 4 26B. Do not point this tutorial’s model ID at CometAPI. Use Google’s official API for hosted Gemma 4 access unless another provider explicitly lists the exact model.

What should you compare?

Use the same prompts and acceptance criteria for local and hosted routes. Record:

• time to first token;
• total response time;
• prompt prefill;
• decode throughput;
• peak memory and SSD reads;
• context length;
• queue and concurrency behavior;
• structured-output validity;
• tool-call validity;
• accepted-task rate;
• retries and human correction;
• total operating cost.

Resident RAM is one metric. It is not the deployment decision.