Everyone Is Benchmarking Edge0-145B on RTX 4090s — I Ran the 35B Tier on a $220 Used Mini PC

qianchuan meng (qianchuan) — Sat, 14 Nov 2026 08:02:26 +0000

All numbers in this post are my own measurements, not official data.
Hardware: Beelink SER (1st gen) mini PC / AMD Ryzen 5 5600H (6C/12T) / 16GB DDR4-3200 single-channel / Vega iGPU / 512G NVMe / Windows 11 x64.
Second-hand price for this exact class (Xianyu listings checked 2026-11-14): ¥1,550–1,590 ≈ $215–220 for a complete unit, ¥1,159 ≈ $160 barebones.

The setup

On Nov 12, Edge0 open-sourced Edge0-145B and Edge0-120B with very provocative numbers:

145B params, 29GB peak memory, 80+ tokens/s decode
Runs on "a suitable laptop", no server needed
Coding, reasoning and agent capability "comparable to frontier cloud models"

Three days in, I went looking for real hands-on tests. There are only a handful, and they're all on RTX 4090/5090 or M2 Ultra (jansverre, goremy, etc.). Chinese platforms are nothing but press rewrites — zero real deployments. Even better: a Japanese blogger already debunked the headline memory claim — the advertised "29GB peak" counts active memory only; real footprint is 54GiB.

Meanwhile, if you're on AMD or an iGPU, this party was never for you: the 145B tier requires NVIDIA CUDA. Your ceiling in the Edge0 family is the 35B tier — the one the official docs say runs on plain CPU.

Cold water first: Edge0-35B is a September preview checkpoint. The model card itself admits weak agent capability, and 145B/120B are the main event. So why bother? Because between "runs on CPU" and "actually usable" sits an order-of-magnitude experience gap, and nobody has quantified it on cheap hardware.

So I did. Here are real numbers from a $220 used mini PC with single-channel RAM and no discrete GPU.

TL;DR: it runs. Warm-cache first token in 1.3s, decode 4.27 tok/s — fine for chat. But prefill is 11.5 tok/s, which makes it mathematically unusable as a coding-agent backend.

Deployment: 4 pitfalls the docs don't mention

Pit 1: HuggingFace is unreachable from China

huggingface.co times out here. The fix is the hf-mirror.com mirror — the model is 18.38GB across 4 shards (4-bit quant + LoRA + pre-router), ~45 minutes with 4 parallel downloads:

curl -L -C - -o model-00001-of-00004.safetensors \
  "https://hf-mirror.com/Edge0/Edge0-35B-A3B-preview/resolve/main/model-00001-of-00004.safetensors"

Keep -C - for resume. Verify byte counts with stat -c%s — all 4 of my shards matched.

Pit 2: The release zip already contains everything

edge0-chat-0.2.0-windows-x64.zip (264MB) from GitHub Releases ships a prebuilt edge0-infer.exe — no compiling — plus the official chat app and the edge0-strata/ engine (for 145B, useless here).

Pit 3: The asset converter crashes on Windows

35B can't use raw HuggingFace weights; you convert them to GGUF assets (+22GB disk). The official converter fails immediately with:

error: [WinError 1314] 客户端没有所需的特权。

Cause: it symlinks source weights into a staging directory, and non-admin Windows without Developer Mode can't create symlinks. Two fixes:

Enable Developer Mode, or
Patch the converter to fall back to a hard link (same volume, no privileges needed, semantically equivalent):

try:
    link.symlink_to(target)
except OSError:
    os.link(target, link)      # hardlink fallback

After the patch: conversion finished in 661s, self-check passed all 703 tensors.

[self-check] edge0-35b.gguf: tensors=703 kv=39 OK
DONE 35b: gguf=20.26GiB lora=40.5MiB took 661.3s

Pit 4: iGPU "Vulkan acceleration" is a negative optimization

The docs support --prefill-device vulkan. On their RTX 4070 SUPER reference machine it cuts time-to-first-token from 17s to 3.9s — tempting.

On my Vega iGPU it's worse across the board: prefill didn't improve (the iGPU is too weak), and decode dropped from 4.27 to 2.93 tok/s — the engine log shows 80 GPU↔CPU handoffs, each one a tax.

Lesson: iGPU users should just stay on CPU.

Bonus: why not just run vanilla Qwen3.6-35B-A3B via Ollama?

Fair question — Edge0-35B is Qwen3.6-35B-A3B plus a LoRA and a pre-router. Two reasons:

Qwen3.6's Gated DeltaNet architecture has a decode-collapse bug on llama.cpp's AMD Vulkan backend — prefill runs 3100+ tok/s, then decode falls off a cliff to single digits. Known issue; my old iGPU would almost certainly hit it.
Edge0's predicted routing (a trained pre-router prefetches the next token's experts from SSD while the current one computes) is a free ~50% decode speedup you don't get on the vanilla model.

On this machine, the Edge0 build is the better deal.

Benchmark: three configurations

Method: 253-token Chinese prompt, 10 warmup steps, 100 timed generation steps, official bench tool.

Config	Decode	Prefill	TTFT	Verdict
CPU 4 threads (default)	4.27 tok/s	11.5 tok/s	22.0s	✅ best
CPU 6 threads	4.20 tok/s	11.3 tok/s	22.5s	no gain
Vulkan prefill + CPU decode	2.93 tok/s	11.0 tok/s	23.1s	worse

The local chat UI I built after deployment — code rendering + metrics chips. This run: TTFT 8.4s, 130 tokens at 3.0 tok/s.

Three counter-intuitive findings:

1. More threads do nothing — the bottleneck is memory bandwidth. 6 threads scored the same as 4. Single-channel DDR4-3200 tops out at 25.6GB/s, and every decoded token streams the active experts' weights through memory. Cores can't help; they're all waiting on RAM.

2. Sparse MoE + predicted routing is why this survives on cheap hardware. 35B total params, ~3B active per token. The engine's pre-router predicts which experts the next token needs and prefetches them from SSD while the current token computes: 3,564 prefetches, only 2 late. This is the core of Edge0's arXiv paper (The Other Half of the Memory Wall).

3. Peak memory was 3.77GB. A 16GB machine never breaks a sweat — expert weights mostly live on SSD and page in on demand. The "phone-class memory" claim is real.

The soul-searching question: can it be a coding-agent backend?

No — and it's arithmetic, not opinion.

OpenCode's agent mode ships a system prompt of roughly 11,000 tokens per request (tool descriptions + role context). My prefill speed is 11.5 tok/s:

11,000 ÷ 11.5 ≈ 16 minutes

That's your wait before the first token appears — before the model even starts "thinking".

The deeper issue: agent tools and SSD-streaming inference are architecturally mismatched. Agents are "huge system prompt + frequent multi-turn calls"; streaming engines shine at "small prompt + long output". Dual-channel RAM would double prefill to ~23 tok/s — 16 minutes becomes 8. Still unusable.

So who is this actually for?

Use case	Experience	Verdict
Local private Q&A (chat UI)	1.3s warm / ~6s cold first token	✅ sweet spot
Document analysis ≤3K tokens	~30s to first token	✅ acceptable
Long-form writing	4.3 tok/s, slower than reading	⚠️ painful
Coding-agent backend	16-minute first token	❌ mathematically dead

If you want the headline speeds, know which tier buys what:

35B tier: no NVIDIA required. The official support matrix is macOS Metal/CPU, iOS, Android CPU, Linux CPU, Windows CPU/Vulkan — no NVIDIA anywhere. The bottleneck is memory bandwidth: my 4→6 thread test was flat, and official numbers scale with bandwidth — M5 16GB does 24.8 tok/s, Android flagships 6–9 tok/s.
145B tier: NVIDIA is a hard gate. Linux/Windows + NVIDIA GPU + CUDA 13, compute capability ≥7.5 (RTX 20-series up) per the v0.2.0 release notes. There's no official "minimum spec", but the model card includes a constrained test run: 16 GiB VRAM + 20 GiB RAM budget, Q3_K, still 44.9 tok/s — the closest thing to a documented floor. On the speed table: RTX 4090 24GB does 91.7 tok/s, RTX 5090 D does 121.2. Also budget disk: 145B weights are 40.8 GiB (Q3_K) / 47.8 GiB (NVFP4).

One line: 35B runs on anything; 145B starts with buying an NVIDIA card.

The junk-reeconomy math

Item	Cost
Beelink SER 5600H/16G/512G (used, 2026-11 market)	~$215
+16GB stick for dual-channel (optional upgrade)	~$28
Electricity (~60W × 4h/day)	~$1/month
Models, engine, tooling	$0 (open source)
Total	~$245 for a fully offline, no-subscription, data-never-leaves local LLM terminal

For comparison: a new RTX 4090 is $1,600+. This budget doesn't buy "fast" — it buys "runs" and "private". And the remaining bottleneck (memory bandwidth) happens to be the cheapest upgrade in consumer hardware.

I still think this deployment proved something: Edge0's claimed floor is real. A GPU-less, single-channel, second-hand mini PC really can run a 35B MoE at interactive speeds. Three years ago that needed a server; today it fits in a shoebox.

What's the biggest model you've run on the worst hardware? Comments welcome.

Declarations: all benchmarks are personal measurements on the stated hardware, not official data; the tested checkpoint is Edge0-35B-A3B-preview. Edge0 name/trademarks belong to the Edge0 team (GitHub: Edge0-AI); this is an independent review — the UI logo in my screenshots is my own design. Inference framework: Apache-2.0; model weights: Edge0 Community Model License 1.0. References: official docs and the edge0-infer repo; patches shown inline.

DEV Community: qianchuan meng (qianchuan)