Since 10 September, four Intel Arc Pro B60 cards in a tower in Meyrin, Geneva, have been running three language models at once — for our own products first. This week we turned that configuration into something you can put in your own building, and opened pre-orders. This post is the spec sheet with the numbers we actually measured, the part that doesn't scale, and what we are not promising.
The problem it solves
We kept hearing the same sentence from law firms, clinics and fiduciaries in Switzerland: "the cloud is not allowed here." Not "expensive", not "we prefer local" — not allowed. For them the choice is not ChatGPT versus something cheaper; it is a machine inside the walls, or nothing.
So SOKKAN Anchor is a box. It runs the models, the retrieval and the gateway on premises. A key can be declared sovereign-only, and the server — not a client-side setting — guarantees that no request under that key will ever reach a cloud provider, not even as a fallback. If the local model is down, you get an error, not a detour.
What is in it
The entry configuration, Anchor E, carries four Intel Arc Pro B60 (24 GB each, 96 GB total) and runs three resident models:
| Card | Model | Format / runtime | Weights | Role |
|---|---|---|---|---|
| 0 | gpt-oss-20b | MXFP4 · vLLM (Intel XPU build) | 13 GB | overnight automation, batch, embeddings |
| 1 | Qwen3-Coder-30B-A3B | GGUF Q4_K_XL · llama.cpp SYCL | 17.7 GB | developers |
| 2 + 3 | Qwen3-Next-80B-A3B | GGUF Q3_K_XL · llama.cpp SYCL | 35.6 GB | finance, HR, management assistants |
All three are MoE models with about 3B active parameters. That is what makes the cohabitation possible: 130B of cumulated weights at the speed of a small model. A single dense 70B would have eaten two cards to be four times slower.
The production platform is an AMD EPYC 9115 (16 cores, SP5) on a Supermicro board with IPMI, 128 GB of DDR5 ECC, two 2 TB NVMe drives in a ZFS mirror, a 1.6 kW ATX 3.1 supply, in a silent office tower or a 4U rack case. Why EPYC rather than a desktop chip: 128 PCIe 5.0 lanes (four cards at full width, no bifurcation games), ECC, and remote management. IPMI is not a luxury for a box we support remotely — it is how we reboot a machine in a law office at 7 am without anyone touching it. ZFS mirror because the box holds the customer's document index, and a dead NVMe must not mean a re-ingestion.
What it does — measured
Every number below was measured on our prototype: a 2018 X299 workstation (i9-9980XE, 64 GB DDR4) that also carries our production job platform — Postgres, twelve Celery workers, two web front ends. It is a pessimistic rig. The EPYC build will not be slower.
Each model alone, warm:
| Model | Decode | Prefill (~4k tokens) |
|---|---|---|
| Qwen3-Coder-30B | 82.4 tok/s | 4,098 tok in 4.2 s |
| gpt-oss-20b | 41.2 tok/s | 3,307 tok in 2.4 s |
| Qwen3-Next-80B | 36.2 tok/s | 4,098 tok in 10.3 s |
All three under load at once — 12 concurrent requests, 4 per model, 300 tokens each: 3,600 tokens in 30.3 s, 119 tok/s aggregate, no errors, no measurable interference between cards. The 20B alone batches to 134 tok/s at 4 requests and 241 tok/s at 8.
Power: 163 W idle with the three models loaded, 211 W peak under the 12-request load, for the four cards. The whole machine is around 350 W. At Swiss electricity prices, running around the clock, that is roughly CHF 750 a year.
Host RAM stayed above 41 GB free at the worst moment — with a full production stack running next to it. That is why the spec sheet does not need 256 GB.
Since 10 September our job platform sends about 107 million tokens a week of batch work — job-ad classification, extraction, enrichment — to this box instead of a cloud provider. That is not a benchmark; it is the workload that paid for the cards.
The part that doesn't scale
The 80B is the model people want: it is the better writer and analyst. It is also the one that does not scale under concurrency, and the reason is the runtime, not the model.
| Concurrent requests | gpt-oss-20b (vLLM) | Qwen3-Next-80B (llama.cpp) |
|---|---|---|
| 1 | 35.1 tok/s | 32.0 tok/s |
| 2 | 35.0 tok/s each | 14.5 tok/s each |
| 4 | 33.6 tok/s each | 7.0 tok/s each |
| aggregate at 4 | 133.9 tok/s | 20.7 tok/s (less than solo) |
vLLM does continuous batching; throughput grows almost linearly with load. llama.cpp does not; its aggregate throughput regresses as concurrency rises. Four users on the 80B get less total work done than one.
Why is it on llama.cpp then? Because vLLM on Intel XPU currently serves only quantisations with a native XPU kernel. MXFP4 gpt-oss has one. Every other quantised MoE we tried routes its expert layers to Marlin — an NVIDIA kernel — and dies with device_capability = -1, whatever you pass to --quantization. Dense AWQ works (int4_gemm_w4a16, and you must set --dtype float16); AWQ MoE does not. So everything that is not gpt-oss runs through llama.cpp SYCL, which is honestly good — Qwen3-Next's hybrid Gated DeltaNet attention runs on it — but serves one user well and four badly.
This goes on the spec sheet as a design rule, not a footnote: the 80B serves two simultaneous users, not a department. Anything real-time — voice, interactive completion — must hit a model served by vLLM. Voice needs roughly 15 tok/s per speaker to stay fluid; the 80B holds two conversations, the 20B at least four.
Two more lessons that cost us real time: models must be loaded sequentially (the RAM peak is at load, not at serve — three simultaneous docker start at boot reproduced a crash), and the first request after loading is 3–4× slower than steady state, so the box warms itself up before a customer touches it.
What we are not promising
- It does not replace ChatGPT or Claude for general use. A developer who compares side by side will prefer the frontier model. Anchor is for the work you are not allowed to send there.
- We have not measured Anchor S (two Arc Pro B70, 64 GB). Its numbers will be published when the cards arrive; until then the page says "estimate".
- vLLM tensor-parallel across the four cards wedged our X299 prototype three times in a day (a load-time DMA/PCIe storm, not RAM — it failed with 59 GB free and production stopped). We will retry on EPYC. It is not needed for the three-model configuration above.
- The larger M (128 GB) and L (192 GB) configurations are described, not open.
What is on top of the hardware
SOKKAN Inference, the gateway from our open-source cockpit (github.com/ninabot-ch/sokkan), runs on the box: one LAN endpoint, OpenAI- and Anthropic-Messages-compatible, so Claude Code, any OpenAI SDK and any agent framework point at it unchanged. Per-key metering, per-key sovereign-only policy, Prometheus/Grafana, WireGuard for support, Ansible for fleet updates. Plus Nina, a document assistant with retrieval over the customer's own files — the same chain as this demo we built for a truck garage's technical manuals.
Price, and how the pre-order works
- Anchor E — 4× B60, 96 GB: CHF 10,450 excl. VAT.
- Anchor S — 2× B70, 64 GB: CHF 11,100 excl. VAT.
- Built to order, delivered and installed in about six weeks, 24-month warranty on new parts. Managed operation (model updates, monitoring, remote support) from CHF 300 a month, optional.
We are a small company and we do not stock EPYC boards and GPUs on the shelf: parts are ordered against a pre-order. The deposit is 20 % (CHF 2,100 for E), fully refundable until the machine ships, deducted from the final invoice. The first five units are sold at this price whatever the GPU market does — B70s went up 26–48 % in August alone — and their serial number is engraved on the front plate.
Page, specs and the full measurement report: ninabot.ch/en/anchor.
If you run Arc Pro cards for inference and hit the same walls — or different ones — I would genuinely like to compare notes. I answer the comments myself.
SOKKAN Anchor is a product of Ninabot Sàrl, Geneva. The Geneva-hosted "Swiss" tier of SOKKAN Inference runs on the same prototype.
Top comments (0)