Since this week, SOKKAN Inference has a tier called "Swiss". It is served from a machine in Meyrin, Geneva, on hardware we own, with no NVIDIA card inside. This post is the honest version of what that machine does, with the numbers we measured — including the ones that don't scale.
Why Intel
We are a small Swiss company. Our inference product already had two cloud tiers, EU-sovereign, served from French datacenters. What we didn't have was an answer for the customer who says "the data cannot leave Switzerland, full stop" — and, longer term, for the customer who says "the data cannot leave the building".
That meant buying GPUs, in 2026, in the middle of a memory shortage. We chased second-hand RTX 3090s for a month and lost every auction above market price. Meanwhile the Intel Arc Pro B60 — 24 GB of VRAM, a 2-slot blower, natively x8 — was still selling near MSRP, at CHF 614 a card. Four of them came to about CHF 2,450 for 96 GB of VRAM. That was the whole argument: VRAM per franc, available now.
The bet was on the software stack, not the silicon. The risk with Intel in 2026 is not the hardware; it is whether vLLM, llama.cpp and the XPU runtime hold up under a service you sell.
What we run
The chassis is a 2018 X299 workstation (i9-9980XE, 64 GB DDR4) that also carries our production job platform — Postgres, Celery workers, a few web front ends. Not a clean benchmark rig. Every number below is therefore pessimistic; the production appliance is an EPYC board dedicated to inference.
Four B60s, each on its own x8 Gen3 root port, run three resident models:
| Card | Model | Format / runtime | Weights |
|---|---|---|---|
| 0 | gpt-oss-20b | MXFP4 · vLLM (Intel XPU build 0.21) | 13 GB |
| 1 | Qwen3-Coder-30B-A3B | GGUF Q4_K_XL · llama.cpp SYCL | 17.7 GB |
| 2 + 3 | Qwen3-Next-80B-A3B | GGUF Q3_K_XL · llama.cpp SYCL | 35.6 GB |
All three are MoE models with about 3B active parameters, which is what makes the cohabitation possible: 130B of cumulated weights at the speed of a small model.
The Swiss tier is card 0.
What it does — measured
gpt-oss-20b under vLLM, warm, on one B60:
| Concurrent requests | Per request | Aggregate |
|---|---|---|
| 1 | 35.1 tok/s | 35.1 tok/s |
| 2 | 35.0 tok/s | 70 tok/s |
| 4 | 33.6 tok/s | 133.9 tok/s |
Prefill is around 5,000 tok/s on a 4.3k-token prompt. In an earlier run at 8 concurrent requests the same card reached 241 tok/s aggregate. Continuous batching does exactly what it says on a model this size: throughput grows almost linearly with load, and per-request speed barely moves.
Power for the four cards: 163 W idle with all three models loaded, 211 W peak under 12 concurrent requests across the three models. Temperatures 52–62 °C. On the UPS, the whole machine — including the production it also carries — moves from 13% to 19% load. At Swiss electricity prices that is roughly CHF 750 a year for the full box, running around the clock.
Tool calling works on all three models. For gpt-oss you need --enable-auto-tool-choice --tool-call-parser openai; without the harmony parser you get zero tool calls and an empty content field, and you will spend an evening wondering why.
What does not scale
The 80B model is the one people want — it is the better writer, the better analyst. It is also the one that does not scale, and the reason is the runtime, not the model.
llama.cpp has no continuous batching. Under concurrency, the same 80B goes from 32.0 tok/s for one request, to 14.5 tok/s each for two, to 7.0 tok/s each for four. Worse: the aggregate throughput regresses — 32 → 28.8 → 20.7 tok/s. Four users on that model get less total work done than one.
So why is it on llama.cpp? Because vLLM on Intel XPU currently only serves quantisations that have a native XPU kernel. MXFP4 gpt-oss does. Every other quantised MoE we tried routes its expert layers to Marlin, an NVIDIA kernel, and dies with device_capability = -1 regardless of the --quantization flag. Dense AWQ works (int4_gemm_w4a16, and you must pass --dtype float16); AWQ MoE does not. Everything that is not gpt-oss goes through llama.cpp SYCL, and llama.cpp SYCL is honestly good — Qwen3-Next's hybrid attention runs on it — but it serves one user well and four users badly.
We also could not get vLLM tensor-parallel across the four cards to stay up on this platform. Loading four workers in parallel wedged the machine three times in one day, once with 59 GB of RAM free and production stopped, so it is not memory; it looks like a DMA/PCIe storm at load on an old chipset. llama.cpp, which loads sequentially, spreads gpt-oss-120b across the four cards without incident — 18.8 tok/s decode, ~250 tok/s prefill, stable — but that prefill number is why you won't see the 120b behind a public tier yet. We will retry TP on the EPYC board.
Two smaller lessons that cost us real time: models must be loaded sequentially (the memory peak is at load, not at serve — three simultaneous docker start at boot reproduce a crash), and the first request after loading is 3–4× slower than steady state, so warm the service before a customer touches it.
The Swiss tier, as sold
Given all that, the Swiss tier is deliberately small and deliberately honest:
- A 20B-class model (gpt-oss-20b), best for chat, RAG and agents. Not a frontier coding model. If you need that, our other tiers are EU-sovereign and better at it.
- Served in Geneva on hardware we own. Data never leaves Switzerland.
- Never rerouted to an EU provider. If the primary model is down, a second model on a neighbouring card takes over, still in Geneva; if that is down too, you get a clear error, not a silent detour.
- Capacity: 4 concurrent requests. Above that, a 429. We would rather tell you than queue you.
- CHF 0.60 / 2.40 per million tokens (in / out), prepaid, metered to the token.
- Anthropic Messages-compatible, so Claude Code and any Anthropic client work unchanged:
export ANTHROPIC_BASE_URL=https://infer.sokkan.ch
export ANTHROPIC_AUTH_TOKEN=sik_your_token
export ANTHROPIC_MODEL=sokkan-swiss
Reselling raw tokens from one card will never be a business — at these prices a saturated B60 earns about what it burns in electricity. The point is different: the same machine hosts services on top of the model (a website assistant for SMEs, a finance assistant for our ERP), and the Swiss tier is the door for developers who want to build the same kind of thing with the same residency guarantee.
What comes next
The same four-card configuration becomes SOKKAN Anchor, an on-prem appliance for organisations that have banned the cloud altogether. The dimensioning rule we learned the hard way goes into the sales sheet: anything real-time — voice, interactive completion — must hit a model served by vLLM; the big llama.cpp model serves two simultaneous users, not a department.
If you are running Arc Pro cards for inference and hit the same walls (or different ones), I would genuinely like to compare notes.
The Swiss tier is live at sokkan.ch/en/inference. SOKKAN is a product of Ninabot Sàrl, Geneva.
Top comments (1)
The honesty about which numbers don't scale is what makes this worth reading — most "we self-host on X" posts quietly drop the concurrency curve. The MoE detail is the real story here: three models at ~3B active each is what lets 130B of cumulated weights cohabit four 24GB cards, and it's why continuous batching gives you near-linear aggregate throughput while per-request speed barely moves. That's a very different economic profile than a dense model of the same footprint.
The line I'd underline for anyone copying this: "the bet was on the software stack, not the silicon." The vLLM/llama.cpp XPU path holding up under a paid service is the actual risk, not the cards.
Two things I'd love numbers on: prefill vs. decode split under mixed prompt lengths (your 5k-tok/s prefill is great, but does it hold with ragged batches?), and what the tail latency looks like at 4+ concurrent when the box is also running Postgres and Celery. Cohabitation is the part everyone underestimates.