Originally published at deepu.tech.
When the whole AI coding frenzy started, I was still skeptical. Like most tech enthusiasts, I tried it but wasn't that impressed by the models' coding capabilities back then. I still used them, but mostly as an autocomplete tool in VS Code. This all changed for me with Opus 4.6 on Claude Code. That was the first time I felt like the model was actually more efficient and smarter than me at coding most of the time. Of course it still needed some hand holding and made silly mistakes at times, but overall I felt the speed of implementing features and fixing bugs was much faster than me doing it alone. I was hooked. I started using it for all my coding tasks and even started to use it for some of my personal projects. I was amazed at how much faster I could get things done with the help of the model.
This was around February or March of 2026. Around six months have passed since, and right now there is a local model running on the laptop implementing a huge feature on LlamaStash (a complex and huge Rust project) as I write this blog post. Now I'm exclusively using local models for most of the coding I do. It's crazy to think that the open source LLMs from Chinese labs that can fit on consumer GPUs have caught up to the frontier models from American companies that need massive data centers to run 😉. And even more impressive is that I can run these powerful models on my laptop without any cloud dependency, which is way greener and wastes less energy. A huge thanks to the Chinese labs for making this possible and for making them open source.
In this post, I will show you how I run Qwen 3.8 on my laptop and how it compares to frontier models like Claude Opus for agentic coding.
Qwen 3.8 27b
When Qwen 3.6 27b was released, I was excited and impressed by its capabilities. I tried some light coding with it. While it was good, it never felt like a replacement for a frontier model like Claude Opus, as it lacked the quality and was slow on top. I was still using Claude/Kimi/GLM etc for most of my coding tasks, with Qwen 3.6 for small offline tasks, housekeeping etc.
But when Qwen 3.8 27b was released, I was blown away by its quality. It was able to understand complex coding tasks and provide accurate solutions. It was also able to understand the context of the code and provide suggestions accordingly. I started using it for some of my main coding tasks and the results were phenomenal. Of course it was still very slow with long prefill times and 10-15 tokens/sec average decode, but personally that wasn't a problem for me since I could give it a task and let it run for a few hours or even overnight, and it finished the tasks at quality on par with Opus 4.6. That seems to be the community conclusion as well: on the Artificial Analysis index charted below, the 27b at xhigh scores 34 against 32 for Opus 4.6.
Most of the coding I do is open source, so the model taking more time than a frontier cloud model wasn't an issue for me. It did take some tuning and setup (more on that below), and it did go on a loop a few times, crashed mid session a few times (probably an issue with my power supply) and would need some hand holding to resume, but overall it was able to complete most of the tasks with minimal intervention. I was impressed by its capabilities and started using it for more complex tasks, with a frontier model like Opus 5 reviewing its output.
Here is the first serious task I gave it and the results I got. It created a plan and one shotted the implementation. See the review comments from Opus 5, they were mostly nits, and Qwen 3.8 27b fixed those in the next commit. I was extremely impressed because the project, LlamaStash, was a complex one and had a huge Rust codebase, but Qwen 3.8 27b was able to understand the codebase and implement the feature with minimal intervention. I started using it for more work and it kept on delivering, and I thought finally the time has come where I no longer need a cloud AI subscription.
Qwen 3.8 Flash Next
Then came Qwen 3.8 Flash Next and oh boy, my mind was blown again. It was a bit better than 27b in terms of quality, but the biggest improvement was overall speed. It was still slow on prefill and had more or less similar decode tok/sec, but it took less time overall to finish tasks as it didn't spend as much time as 27b thinking. Here is a real world sample.
The numbers make the point better than the feeling does. Flash Next does not decode faster, it just doesn't overthink as much as 27b. The top two rows are a short greedy prompt, the rest are five coding tasks with the generated code executed against hidden tests.
| Flash Next | 27b ROCmFP4 | |
|---|---|---|
| Decode | 23.7 tok/s | 24.5 tok/s |
| Prefill | 96 tok/s | 150 tok/s |
| Correctness, 5 tasks | 5/5 | 5/5 |
| Tokens spent | 2,320 | 4,223 |
| Wall clock | 76.5 s | 289.8 s |
Same answers, 45% fewer tokens, 3.8x faster to done, but 27b is the one that is ahead on both raw speed numbers.
I would say Flash Next is on par with Opus 4.8 in terms of quality, just from experience so far (They have an AA score of 40 and 42 respectively). So thanks again Alibaba and Qwen team. It did need a lot more memory to run on my 128GB Strix Halo, around 86 GiB resident against 31 GiB for the 27b, so it isn't the only choice yet. I use it for the more complex tasks and the ones that need to get done faster. For everything else 27b is still my go to, because at 31 GiB I can keep two of them loaded at once and switch thinking modes without a restart, and as the chart shows they aren't far apart in quality.
My current setup
Hardware: Strix Halo
I wrote a detailed post about my fully offline AI assisted Linux development machine. TL;DR; It's an ASUS ROG Flow Z13 with Ryzen AI Max+ 395, 16 cores and 32 threads, AMD Radeon 8060S integrated GPU with 40 compute units and 128GB unified memory. I run Arch Linux on it with Niri + DMS as my desktop.
Orchestrator: LlamaStash
I use my tool LlamaStash to orchestrate the model and manage the sessions. It is a fast TUI, CLI, daemon, and OpenAI-compatible proxy for running local LLMs via backends like llama.cpp and vLLM. It has a lot of features that make it easy to run local models and manage them, like multiple backends, preset files, named launches, auto start etc.
I mainly run llama.cpp as backend with different builds for different models. Here is my LlamaStash config for it.
backend:
llamacpp:
# llama-server build variants; first entry is the default binary.
servers:
- binary: /mnt/work/Workspace/llms/llama.cpp/build-hip/bin/llama-server # ROCm0
- binary: /mnt/work/Workspace/llms/llama.cpp/build-vulkan/bin/llama-server # Vulkan0
# Shim, not raw binary: sources q38rocm's setup_env.sh (RADV_PERFTEST,
# unified memory, ICD pin) that llamastash has no env hook for.
- binary: /home/deepu/.local/bin/q38rocm-llama-server
name: ROCmFP4 # ROCm0 + Vulkan0
# unslothai fork: only needed to load qwen4exp MTP draft heads, which stock
# rejects (~1.3x speedup). Drop when MTP support lands upstream.
- binary: /mnt/work/Workspace/llms/llamacpp-unsloth/llama-server
name: UnslothMTP # ROCm0
# Fork build for the DFlash2 sidecar drafter; --spec-draft-adaptive was
# never upstreamed.
- binary: /home/deepu/.local/bin/dflash-llama-server
name: DFlash2 # Vulkan0
ds4:
enabled: true
servers:
- binary: /mnt/work/Workspace/llms/ds4/ds4-server
vllm:
enabled: true
servers:
- binary: /home/deepu/.venvs/vllm/bin/vllm
lemonade:
enabled: true
Four llama.cpp builds for one machine looks silly until you need them. Stock ROCm is the default, Vulkan wins decode on the ROCmFP4 fork, and the two named forks exist only because their speculative decoding paths were never upstreamed (at the time of writing). A LlamaStash preset can pin which build it wants, so I never have to remember which is which.
Harness: Pi
For coding I mostly use Pi as harness these days. It pairs well with Qwen models and I have it setup to follow the same rules and memories as my, hopefully getting closer to retire, Claude Code setup. Below is the LlamaStash provider block in ~/.pi/agent/models.json, which llamastash integrations writes for you.
{
"providers": {
"llamastash": {
"api": "openai-completions",
"apiKey": "!llamastash api-key",
"baseUrl": "http://127.0.0.1:11435/v1",
"name": "LlamaStash",
"models": [
{
"id": "Qwen3.8-Flash-Next-UD-Q4_K_XL",
"name": "Qwen3.8-Flash-Next",
"contextWindow": 131072,
"maxTokens": 8192
},
{
"id": "Qwen3.8-27B-Q8_0",
"name": "Qwen3.8-27B-Q8_0",
"contextWindow": 131072,
"maxTokens": 8192
},
{
"id": "Qwen3.8-27B-ROCmFP4-FAST",
"name": "Qwen3.8-27B-ROCmFP4-FAST",
"contextWindow": 131072,
"maxTokens": 8192
},
{
"id": "Qwen3.8-27B-UD-Q6_K@xhigh",
"name": "Qwen3.8-27B (xhigh)",
"contextWindow": 131072,
"maxTokens": 8192
},
{
"id": "Qwen3.8-27B-UD-Q6_K@medium",
"name": "Qwen3.8-27B (medium)",
"contextWindow": 131072,
"maxTokens": 8192
}
]
}
}
}
The last two entries are the interesting ones. A <model-id>@<name> id pins the request to one specific running launch instead of letting the proxy pick, which is what puts both thinking modes in the model picker at the same time. Pi passes custom provider ids through to the API unchanged, so the @name arrives at LlamaStash verbatim.
Those two I add by hand. llamastash integrations only writes a row per model on disk, it does not know what you are going to name your launches. There is an open issue to teach the patcher about named launches.
Model Quants
I did some research and benchmarks to settle on the below quants for Qwen 3.8 27b and Flash Next. I found that these quants give the best balance between speed and quality for my use case.
- Qwen3.8-Flash-Next-UD-Q4_K_XL (104 GB): For best quality and faster execution but single agent. Needs the unslothai/llama.cpp fork to load its MTP draft heads, stock llama.cpp rejects them (at the time of writing).
- Qwen3.8-27B-UD-Q6_K (20 GB): Default model for multi agent work
- Qwen3.8-27B-Q8_0 (27 GB): For tasks that need more quality and can take longer to finish
- Qwen3.8-27B-ROCmFP4-FAST (14 GB): For one-shot tasks that need to finish faster and can compromise a little on quality. This one needs the julianmb/q38rocm fork, it will not load on stock llama.cpp.
Decode on this hardware is memory bandwidth bound, not compute bound. At 7.27 tok/s on a 27.1 GiB file the GPU is pulling about 212 GB/s against the roughly 256 GB/s the LPDDR5X tops out at, so it is running at 83% of theoretical peak. That is why the sizes above track the speed ordering almost exactly. Smaller file, fewer bytes per token, faster decode.
Optimizing Qwen 3.8 for speed and efficiency
Config tuning with presets
For all the models, I use 128k context rather than full 256k to avoid very long prefills and slow downs at long context. I also use the following settings for all the models to get best performance and quality. The settings are applied using LlamaStash presets so that I can easily switch between different models and settings via TUI or CLI and can reproduce the settings easily.
presets:
# qwen35 arch: hybrid SSM, full attention every 4th layer, one embedded MTP
# layer at blk.64, native ctx 262144. Draft n=5 measured best, see below.
Qwen3.8-27B-*:
default: coding-xhigh
entries:
coding-xhigh:
knobs:
mtp: true
mtp-draft-n: 5
ctx-size: 131072
n-gpu-layers: 99
flash-attn: true
no-mmap: true
parallel: 1
extras:
- --reasoning-format
- deepseek
- --reasoning-preserve
- --reasoning-effort
- xhigh
- --cont-batching
- --spec-draft-p-min
- "0.0"
- --temperature
- "0.0"
- --presence-penalty
- "0.0"
- --repeat-penalty
- "1.0"
A wildcard key like Qwen3.8-27B-* covers every quant of the model, so Q6_K, Q8_0 and the rest all inherit one config. coding-medium is the same block with --reasoning-effort medium, and I will come back to that below.
Two of those values are important.
mtp: true turns on Multi Token Prediction, the speculative decoding head that ships inside the GGUF itself. It is worth 2.3x to 3.1x on decode depending on the build, and nothing else in the whole config comes close. The model verifies every drafted token, so on stock llama.cpp the output does not change, only the speed.
--temperature 0.0 is deliberate, and it goes against Qwen's own guidance, which asks for temp 1.0 / top-p 0.95 / top-k 20 in thinking mode. I measured the official sampling and it cost 2.8x wall time, 772 seconds against 273 on the same two questions, because MTP draft acceptance collapses from 59.6% to 45.7% when the sampler stops agreeing with the drafter. The vendor spec assumes you are not running speculative decoding. Greedy also makes generation byte reproducible, which is what let me tell a real difference between two configs from a lucky roll.
Flash Next gets its own preset, mostly because it has to be pinned to the fork build that can load its draft heads.
Qwen3.8-Flash-Next-*:
default: coding-xhigh
entries:
coding-xhigh:
server: llamacpp-UnslothMTP
knobs:
ctx-size: 131072
flash-attn: true
mode: chat
mtp: true
mtp-draft-n: 4
n-gpu-layers: 99
parallel: 1
extras:
- --reasoning-format
- deepseek
- --reasoning-preserve
- --reasoning-effort
- xhigh
- --cont-batching
- --spec-draft-p-min
- "0.0"
- --temperature
- "0.0"
- --presence-penalty
- "0.0"
- --repeat-penalty
- "1.0"
⚠️ Never set
no-mmapon Flash Next. The model has a 26.8 GiB per-layer token embedding table that llama.cpp streams lazily from the file mapping, and it is read on every single token. Turning mmap off makes the loader pull the whole thing into anonymous memory instead, which is the wrong direction on a box with about 20 GiB of headroom. On the 27b, which has no lazy tensor,no-mmapis harmless.
Named launches
LlamaStash has a --name option that lets you run multiple launches of the same model at the same time. Each --name is its own process with its own KV cache and presets. This is useful for running multiple agents in parallel, each with its own context and reasoning mode for example. From the TUI, Alt+Enter prompts for the name instead.
💡 Named launches landed in LlamaStash 0.3.0. Before that, a second launch of the same model would start fine but was unaddressable, because the proxy handed every request to the first Ready launch it walked past.
Reasoning effort switch with presets
I have presets duplicated with both xhigh and medium reasoning effort so that I can switch them via Pi's model selector using LlamaStash named launches. The reasoning effort setting steers the models at the chat template level. medium has no steering. xhigh injects a "think carefully" instruction. low injects a "keep your thinking brief" instruction.
xhigh(the template default): "Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."low: "Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."medium: nothing at all. There is no branch for it in the template, so the instruction string stays empty.
The model truly obliges these short instructions and the difference is very obvious in its output.
Reasoning effort is a server flag, not a per-request one. One llama-server process serves exactly one mode, so switching between xhigh and medium used to mean restarting the model and waiting out a cold load. Named launches fix that: run the same model twice under different presets, and pick the mode from Pi's /model picker.
llamastash run Qwen3.8-27B-UD-Q6_K --preset coding-xhigh --name xhigh
llamastash run Qwen3.8-27B-UD-Q6_K --preset coding-medium --name medium
Now Qwen3.8-27B-UD-Q6_K@xhigh and @medium both show up in Pi. @medium for the bulk of the work, @xhigh when a task needs the model to actually stop and think, switched mid-session without a restart.
Parallel coding agents
I can also use a planner agent to spawn multiple coder agents with different reasoning effort and context, addressing them as Qwen3.8-27B-UD-Q6_K@planner, @coder-one and @coder-two. This is very useful when I want to run multiple agents in parallel for a complex task.
llamastash run Qwen3.8-27B-UD-Q6_K --preset coding-xhigh --name planner
llamastash run Qwen3.8-27B-UD-Q6_K --preset coding-medium --name coder-one
llamastash run Qwen3.8-27B-UD-Q6_K --preset coding-low --name coder-two
Three Q6_K launches at 128k is roughly 93 GiB of the 125.5 GiB available, so three is the comfortable ceiling on this machine.
What about --parallel instead?
One server with --parallel N also serves N sessions off a single copy of the weights, and it looks like the cheaper option. It is cheaper, and on a current llama.cpp it is also the faster one. It is a good option when you want to run multiple sessions using the same model preset. It still does not replace the setup above, but the reason is narrower than I first thought.
Three sessions, 30k of context each, 1024 tokens out, Q6_K with MTP, per-session window pinned to 40960 in both arms. Wall clock is the cold turn, the one that starts a session:
| Three sessions at 30k | --parallel 3 |
3 named launches |
|---|---|---|
| Prefill, aggregate | 453 tok/s | 146 tok/s |
| Decode, average per session | 5.7 tok/s | 5.3 tok/s |
| Effective decode | 4.95 tok/s | 3.24 tok/s |
| Cold turn, wall clock | 557 s | 815 s |
| Tokens generated | 2,755 | 2,640 |
| Draft acceptance | 0.59 / 0.62 / 0.59 | 0.55 / 0.53 / 0.59 |
| Memory | 30.8 GiB | 70.6 GiB |
Context each at -c 131072
|
43.8k | 128k |
| Thinking modes available | one, server wide | one each |
The gap is prefill, not decode. Continuous batching interleaves three prompts on one server at 453 tok/s aggregate, while three separate processes thrash each other down to 46-53 tok/s each, so the same 30k window takes 164-326 s batched against 569-645 s split. Decode itself is near enough a tie, 5.7 against 5.3 tok/s per session. It is the prefill that puts --parallel 3 at two thirds the wall clock on less than half the memory, for 4% more tokens generated. Effective decode in the table is tokens generated divided by wall clock, which is the number you actually sit through. Follow-up turns favour it too, by roughly 1.7x, though I measured those on the previous llama.cpp build.
So the case against it is just the last two rows. The context divides rather than shares: with unified KV off, n_ctx_seq = n_ctx / n_parallel, so -c 131072 --parallel 3 leaves each session 43,776 tokens instead of 128k. Turning --kv-unified on gives every session the whole window back for no extra memory, but it cost 66% more cold-turn time at three slots when I measured it, so I leave it off. And reasoning effort is a server flag, so one --parallel 3 launch gives every session the same thinking mode, which is the whole point of the setup above. Named launches are what I use to keep xhigh and medium both addressable, and --parallel is what I would reach for to run several sessions on one preset.
One more thing worth knowing: slot affinity is a heuristic, not a binding, so a request arriving while its own slot is busy lands on another session's slot and evicts its cache.
⚠️ Update your llama.cpp before trusting any of this. On b10803 the
--parallelarms were not merely slower, they were broken: three sessions cold-prefilling 30k at once came back after generating 2, 586 and 36 tokens with draft acceptance at 0.000, against a clean 1024 each from separate launches. That is wrong output, not slow output. On b10888 the same test gives healthy acceptance on every stream and the table above. The separate-launch arms barely moved between the two builds (815 s against 797 s).
Caveats: one run per cell, one machine, one model, llama.cpp b10888. Upstream is actively changing this area, so re-measure after a bump rather than trusting the table.
Some benchmarks
A caveat before the tables. These come from three separate measurement sessions on different builds over about three weeks, so read each table on its own and do not cross reference cells between them. Everything is on the same Strix Halo laptop, on AC, with the power profile pinned to performance, greedy sampling, and decode and prefill taken from llama.cpp's own timings block rather than from wall clock.
First, the 27b on a short prompt with an empty window. This is the best case number, it makes MTP look like magic.
| Build and quant | Decode, MTP off | Decode, MTP on | Prefill (4k prompt) | Draft acceptance |
|---|---|---|---|---|
| Stock ROCm, Q8_0 (27 GB) | 7.3 t/s | 22.4 t/s (n=5) | 270 t/s | 73% |
| Stock Vulkan, Q8_0 (27 GB) | 7.4 t/s | 22.6 t/s (n=5) | 201 t/s | 73% |
| ROCmFP4 fork, FAST (14 GB) | 13.0 t/s | 29.9 t/s (n=6) | 259 t/s | 84% |
MTP is a 3.1x win on Q8_0 and 2.3x on the ROCmFP4 build. A draft length of 5 beat the backend default of 3 at every context length I tested, with no crossover.
Now the same thing with the window actually full, which is the number that matters for agentic coding. Every row here has MTP on.
| Context | Q8_0 prefill | Q8_0 decode | ROCmFP4 prefill | ROCmFP4 decode |
|---|---|---|---|---|
| empty | 270 t/s | 22.4 t/s | 259 t/s | 29.9 t/s |
| 32k | 210 t/s | 15.6 t/s | 193 t/s | 19.6 t/s |
| 64k | 164 t/s | 14.2 t/s | 124 t/s | 16.6 t/s |
| 128k | 115 t/s | 11.3 t/s | 66 t/s | 13.2 t/s |
| 256k | 71 t/s | 5.4 t/s | failed | failed |
MTP's payoff shrinks as the window fills, from 3.1x on an empty window down to 1.15x at a full 256k, because the verify pass gets more expensive as the KV cache grows even though acceptance holds up fine. And the ROCmFP4 fork's advantage is decode, not prefill. It falls behind stock Q8_0 on prompt processing by 128k. That, plus the prefill times, is the real reason I cap the window at 128k instead of running the native 256k.
Here is what one actual Pi turn looks like on the Q6_K I use day to day, pulled straight from the server log.
prompt eval: 181633 ms / 31410 tokens (172.9 t/s)
eval: 6231 ms / 89 tokens ( 14.1 t/s)
draft acceptance = 0.835 (71 accepted / 85 generated), mean len = 5.18
Three minutes of prefill on a 31k token transcript, then 14 tok/s of decode. The follow up turn only had to prefill the 4,770 new tokens and came back in 45 seconds. That gap between a cold session and a warm one is the single biggest thing that shapes how these models feel to work with.
And the 27b against Flash Next on five real coding tasks, with the generated code executed against hidden tests.
| Model | Correctness | Tokens | Wall clock |
|---|---|---|---|
| Qwen3.8-Flash-Next | 5/5 | 2,320 | 76.5 s |
| Qwen3.8-27B ROCmFP4 FAST | 5/5 | 4,223 | 289.8 s |
Both got everything right. Flash Next just used 45% fewer tokens to say the same thing. On one task the 27b spent 1,784 tokens where Flash Next spent 424. Five tasks is not a correctness ceiling though, it measures verbosity, and a harder suite would be needed to separate them on capability.
What did not help
About as useful as the wins, and it saved me from baking a few of these into my config:
- Qwen's official sampling. 2.8x slower, covered above.
-
turbo4for the V cache on ROCmFP4. 49% slower at 10k context, and it changed what the model generated: more thinking, shorter answers. -
--reasoning-budget. Completely inert. Dropping it from 8192 to 2000 produced byte identical output, because the model stops well under the budget anyway. The client'smaxTokensis the cap that actually binds. -
repeat-penalty 1.05. The one config arm that got facts wrong in its answers. Since greedy output is deterministic, that is a reproducible property of the setting rather than a bad roll. Upstream warns that penalties "can force rare-token gibberish in long generations", so I left it at 1.0. -
GPU clock pinning. Writing
hightopower_dpm_force_performance_levelsilently hard hung the machine about 25 seconds later and needed a power cycle. Do not try this on Strix Halo. Auto mode already boosts to 1900-2500 MHz under load anyway.
The two knobs that did earn their place: --ubatch-size 512 over 1024, worth about 4% on cold prefill and reproduced across rounds, and running the ROCmFP4 fork on Vulkan0 instead of ROCm0, worth 24% on decode.
Caveats
These are the issues and learnings so far from using these models.
Disclaimer: A lot of my coding sessions were not run under ideal conditions. I'm currently on vacation and I didn't bring the laptop's dedicated power brick that supplies 180W peak. Instead I have a 100W PD GaN charger that cannot sustain a full GPU load, so I had GPU resets and battery drain shutdowns causing model restarts. I'm confident the overall experience will be better than what I had. I will test this setup with the proper power brick when I get back and will update this post with the findings.
- The most annoying issue, IMO, is the prefill times. The first prompt on cold start (session start, model restart, after compaction etc.) takes a long time to process. On my setup a 31k token transcript takes about 3 minutes, and a genuinely full 128k window is closer to 18 minutes at the 115 tok/s that table above shows. Subsequent prompts are processed almost immediately due to cache and decode is 10 to 15 tok/s, which is not bad at all, since you see stuff happening and it doesn't feel that slow.
- Define the same context length in the Pi model definition as the launch, to avoid abrupt compaction. With the proper context length Pi will compact more efficiently and you lose less context.
-
Set the client's
maxTokensto at least 8192. This is the single most impactful setting and it lives in the client, not in the server preset. Below that, the model burns the whole budget thinking, returnsfinish_reason: length, and you get an empty string after eight minutes of waiting. I lost a whole evening to this before I worked out where it was coming from. -
Thinking is 90 to 95% of the generated tokens. One 40k context question produced 29,006 characters of thinking and 1,426 characters of answer. That ratio, not the decode rate, is what actually sets how long a task takes. That is why
mediumis so much faster thanxhigh, and why it is my default for most tasks. -
The ROCmFP4 FAST build has no prompt caching. Every request reports
cache_n: 0, so repeated prompts pay a full prefill each time. Stock llama.cpp caches the prefix. In an agent loop that mostly appends to a transcript, that turns the fastest-decoding build into the slowest one to actually finish. It is why the FAST build is a special-case tool for one-shots and not the default. -
The fork's speculative path is not strictly greedy by default. It logs
Qwen MTP strict verification is disabled; greedy output may diverge, and completion lengths drift for an identical prompt (152, 158 and 160 tokens across three runs). Turning--spec-mtp-strict-qwenon costs nothing measurable, 1135 seconds against 1136, so there is no reason to leave it off. - Not really a caveat: Strix Halo is memory bound, not GPU clock bound, for these models. This changes which knobs are worth touching. Decode runs at about 83% of the memory bus's theoretical peak, so it barely responds to power. Prompt processing is the compute bound half and does respond. Sweeping the laptop's power profile, prefill gained 12.7% going from 55W to 90W while decode gained only 4.2%, and most of that came in the first step from 55W to 70W. Past 70W the returns flatten, which is where I leave it. This particular sweep was measured on Qwen3.6-27B-Q6_K, so read the ratio rather than the absolute numbers.
| Power profile | Prefill (pp4096) | Decode (tg256) |
| ------------- | ---------------: | -------------: |
| 55W | 284.4 t/s | 8.78 t/s |
| 70W | 307.6 t/s | 8.89 t/s |
| 90W | 320.6 t/s | 9.15 t/s |
Is it viable
So can Qwen 3.8 running on your laptop really replace Claude Opus for Agentic coding? It 100% can if you are patient. If you want results fast it's probably not for you, but if you don't mind it taking 2 or 3 times longer than a frontier cloud model to finish a task, and you are willing to spend some time setting it up, then yes, it can replace Claude Opus 4.6 to 4.8 for agentic coding and for use with tools like Hermes, which I plan to run alongside Pi. I will be using these as my primary coding models until the next best local model comes out. Can't wait for Qwen 4 to drop, I have a gut feeling that we will have an Opus 5 class model that can run comfortably on consumer GPUs (I mean you can already run Kimi K3 but it isn't practical for most people). I will be testing it as soon as it comes out and will share my findings here.
If you like this article, please leave a like or a comment.
Top comments (0)