DEV Community: David

We Benchmarked Our Agent Against opencode: Same Task, Same Model, 40 Percent Fewer Credits

David — Sun, 23 Aug 2026 00:11:55 +0000

Every coding agent says it is efficient. Almost none of them publish the bill. So we ran the boring experiment: the same bugfix, the same model, the same API, the same prices, and a byte identical prompt, once through opencode and once through the coding agent inside Locally Uncensored.

Headline: opencode averaged 2157 credits over three runs. Our 2.6.6 agent finished the identical task for 1298. That is about 40 percent less, and even the cheapest opencode run came in 29 percent above our number.

The interesting part is not the headline. It is why the gap exists, and it is not the reason most people guess.

Setup

A cost comparison is only worth reading if everything that drives cost is nailed down. What was held constant:

Held constant	Value
Task	Fix a failing test in a small npm repo, then commit
Repository	Three files, a one line bug in `add.js`, tests red at the start
Prompt	Byte identical, sha256 `29cec6c3...cf62687`
Model	`deepseek-ai/DeepSeek-V3.2`
Endpoint	The same OpenAI compatible API for both agents
Prices	Same account, same tier, same per token rate
Counting	One wire proxy in front of the API, credits read before and after every run
opencode	1.18.21 from npm, wired as an OpenAI compatible provider, `opencode run --auto`, otherwise defaults

Success was defined before the runs, not after:

npm test passes
exactly one commit, with the required message
only add.js changed
clean working tree at the end

All four runs cleared that bar. Nothing failed, so cost is the only variable that moved.

The numbers

Run	Credits	Requests	Prompt tokens	Success
opencode, run 1	1679	8	98,789	yes
opencode, run 2	2433	11	146,058	yes
opencode, run 3	2358	11	146,387	yes
Locally Uncensored 2.6.6	1298	16	74,629	yes
Locally Uncensored 2.6.5	4395	30	257,270	yes

Read the last row first. Our own shipped agent from one release earlier is the most expensive thing in that table, by a lot. This is not a chart built so that we win by construction. It is a chart that shows what one efficiency pass is worth, and the previous version of our own software is the loser in it.

Why the gap exists

The tempting explanation is that one agent is smarter and needs fewer steps. That is not what happened, and the direction is the reverse of what you would expect.

opencode used fewer turns than we did. Eight to eleven requests against our sixteen. If you scored this on steps, opencode wins. The bill went the other way because of what every single request carries.

Per request	opencode	Locally Uncensored 2.6.6
Prompt tokens per request	12,349 to 13,308	4,664
Tool catalogue size	21,188 bytes	7,703 bytes
Credits per prompt token	0.01700 / 0.01666 / 0.01611	0.01739

That last row is the honest one. The billing rate is the same. Credits per prompt token land within a few percent across all four runs, and ours is marginally the highest of the set. Nobody got a secret discount. The entire difference in the invoice is token volume, not token price.

Two things drive that volume, and both are familiar to anyone who has built an agent loop:

The fixed block. A tool catalogue of 21,188 bytes against 7,703 bytes is roughly three times the standing overhead, and you pay it again on every call in the loop, whether the model touches those tools or not.
Context decay. As the agent works, the transcript grows. Old tool output that stopped mattering ten steps ago keeps getting resent at full length unless something actively trims it.

Put together, a fixed block that size pushes every opencode request past 12,000 tokens. Six agent steps at that weight already approach our total consumption for the entire task.

What changed between 2.6.5 and 2.6.6

The 4395 in the table is not a strawman we built for the article, it is what we shipped in the previous release. Between 2.6.5 and 2.6.6 we went after exactly the two items above: how big the fixed block is, and how much of the transcript gets resent. Measured over the same set of tool driven runs, that cut credit consumption by 78.6 percent, and on the longest run in the set by 80.4 percent. The opencode comparison is simply what fell out when we pointed the same measurement at somebody else's loop.

Limits of this benchmark

This is where vendor benchmarks usually go quiet, so here is how far the number actually carries.

One scenario. A tiny repo and a one line bug. It says nothing about a large codebase, a multi file refactor, or a session that runs for an hour. We did not run those.
Uneven sample. opencode ran three times, we ran once. The spread inside opencode alone is 45 percent, from 1679 to 2433. Our own spread is unknown. One run is a data point, not a distribution.
Default settings. opencode ran as it ships. It is configurable, and a tuned setup with a trimmed tool set would land somewhere else. We did not tune it in either direction.
opencode is free software. The tool costs nothing. Everything measured here is the model bill, which you pay to whichever provider you point at. This is token efficiency, not licence fees.
Cost is not quality. Every run in the table produced correct, committed work. On a harder problem the ranking could look different, and cheapest is never automatically best.

What the measurement does support is a narrower claim than the headline: on short, well scoped agent tasks, opencode can hardly land below us, because the fixed per request overhead sets a floor. Even its best run, with only eight requests, still needed 98,789 tokens.

Credit where it is due

opencode finished the job three times out of three, took fewer turns than we did, and produced a clean diff with the right commit message every time. It is a genuinely good agent and it is open source. Nothing here is an argument to stop using it.

It is an argument to measure your own loop. Agent bills are made of tokens you never see, and two tools that both feel fast can be a factor of 1.66 apart on the invoice. If you build agents, the two numbers worth putting on a dashboard are prompt tokens per request and the byte size of your tool catalogue. They predict the bill better than step count does.

Full writeup with the methodology and the raw counts: opencode Alternative: We Measured the Cost per Task. The agent lives inside Locally Uncensored (AGPL, free), and the hosted models we benchmarked against sit in LU Labs Cloud if you want them in one picker. You can also point it at a model on your own GPU and skip the API bill entirely.

FAQ

Is opencode expensive? opencode is free. The bill is the model. On this one line bugfix with DeepSeek V3.2 the three runs cost 1679, 2433 and 2358 credits, a 45 percent spread between cheapest and dearest.

Why does it use so many tokens? Not through extra steps, it used fewer than we did. Each request carries more: 12,349 to 13,308 prompt tokens against our 4,664, with a tool catalogue of 21,188 bytes against 7,703 resent on every call.

Is the billing rate really identical? Yes, and that is the point. Credits per prompt token came out at 0.01739 for us and 0.01700, 0.01666, 0.01611 for opencode. The gap is volume, not price.

Can I reproduce it? Three file npm repo with a red test, prompt pinned at sha256 29cec6c3...cf62687, model deepseek-ai/DeepSeek-V3.2, opencode 1.18.21 at defaults, requests counted through a wire proxy, credits read before and after each run.

Run Qwen 3.8 27B Locally: Real GGUF Sizes, the KV Cache Trick, and the Template Trap

David — Fri, 14 Aug 2026 20:48:03 +0000

Qwen 3.8 arrived as two different releases with two different licences, and only one of them is something you can put on a card you own. The 2.4 trillion parameter A95B opened up on 12 August under Alibaba's own qwen3.8-max terms. The one that matters for local work is Qwen 3.8 27B, whose safetensors went up on 13 August at 08:23 UTC with an Apache 2.0 LICENSE file following the next morning. Both dates are off the Hugging Face commit log, not a launch post.

Here is the practical picture: what it needs, why its long context is unusually cheap, and the one setting that makes people think they downloaded a broken quant.

The shape of the model decides everything

27B dense parameters across 64 layers, hidden size 5120. The interesting part is in config.json, where layer_types reads 48 linear attention layers and 16 full attention layers, alternating three to one (full_attention_interval: 4). Only those 16 layers keep a KV cache.

The rest of the shape: 24 attention heads with head_dim 256 and 4 KV heads, a 248,320 token vocabulary, and max_position_embeddings of 262,144. It is a native vision language model, so images and video go in without a wrapper, and the ggml-org pack also ships a multi token prediction head as a separate file.

The numbers

Sizes below are the file sizes Hugging Face reports for unsloth/Qwen3.8-27B-GGUF, read on 14 August 2026. Packs differ by a few hundred megabytes, so check the repo you actually pull from. lmstudio-community has Q4_K_M at 16.8 GB and ggml-org at 19.0 GB for the same nominal quant.

Quant	Size on disk	Realistic home
UD-IQ2_XXS	9.0 GB	12 GB cards, visible quality cost
UD-Q2_K_XL	10.7 GB	12 GB cards, almost no context left
UD-Q3_K_XL	13.4 GB	16 GB cards
Q3_K_M	13.8 GB	16 GB cards
IQ4_XS	15.7 GB	largest quant that stays whole on 16 GB
Q4_K_M (sweet spot)	17.1 GB	24 GB cards
Q5_K_M	19.8 GB	24 GB, less context headroom
Q6_K	22.9 GB	24 GB barely, or 32 GB
Q8_0	29.0 GB	32 GB or a two card split
BF16 (from `ggml-org`)	53.8 GB	server cards, or CPU and patience
mmproj	0.9 GB	the vision encoder, separate file

Machine classes, honestly:

24 GB GPU: Q4_K_M whole, with real context headroom. This is the card the model was sized for.
16 GB GPU: IQ4_XS fits whole; Q4_K_M works with a few layers offloaded and costs you speed.
12 GB GPU: only the 2-bit quants, and you will feel it. A 3060 runs it, slowly.
Apple Silicon: 32 GB unified memory is comfortable at Q4_K_M, 24 GB works if nothing else is open.

Where the hybrid layout pays off

Every full attention layer stores a KV cache that grows with the context. At fp16 one token costs 2 x 4 heads x 256 dim x 2 bytes = 4 KB per layer. A conventional 64 layer model pays that on all 64 layers, which is 256 KB per token. Qwen 3.8 27B pays it on 16 layers, so 64 KB per token.

Context	KV cache	Q4_K_M total
8K	0.5 GB	17.6 GB
32K	2.0 GB	19.1 GB
128K	8.0 GB	25.1 GB
262K	16.4 GB	33.5 GB

That is the difference between a long window on the spec sheet and one you actually turn on. A 128K session that would need 32 GB of cache on a dense model needs 8 GB here.

With a current llama.cpp:

llama-server -m Qwen3.8-27B-Q4_K_M.gguf \
  --jinja \
  -ngl 99 \
  -c 32768

The template trap

That --jinja flag is not optional, and it is the single biggest source of "this quant is broken" reports.

Qwen 3.8 ships its own chat template. Load the model without it and the model has no reliable marker for where your turn ends and its answer begins. The two failure modes both look like a bad conversion: either it rambles past the stop token, or it answers in a clipped voice and loses the conversation between turns.

There is a second, sharper version of this. The official template wraps each assistant turn in a think block even when the reasoning is empty, then opens another one when generation starts. Across several turns those nest and the history gets truncated. Several GGUF packs already ship a corrected chat_template.jinja. If you converted the weights yourself, swap the template before you blame the quantization.

Vision needs the second file

The vision encoder is not inside the language GGUF. Download the mmproj file, about 0.9 GB, and load it alongside:

llama-server -m Qwen3.8-27B-Q4_K_M.gguf \
  --mmproj mmproj-F16.gguf \
  --jinja -ngl 99

Skip it and you have a strong text model that will politely tell you it cannot see the image you just pasted.

Thinking is a dial, not a switch

The model reasons before answering by default. Lower reasoning_effort from the default to medium or low for quick answers, or pass chat_template_kwargs with enable_thinking: false to skip the reasoning pass entirely. Locally the cost of thinking is not money, it is your own time watching tokens arrive, which is the more annoying currency.

The no-terminal route

If you would rather click than type flags, Locally Uncensored (open source, AGPL) wraps this: install, open the Model Manager, paste a Qwen 3.8 27B GGUF repo, pick the quant that fits your card, chat. It carries the llama.cpp engine, handles the offload split and the template, and keeps everything on your machine with no account and no telemetry.

And the 2.4T reality check

The big Qwen 3.8, Qwen/Qwen3.8-2.4T-A95B, has open weights but not an open licence. Hugging Face reports license: other with license_name: qwen3.8-max, so read the terms before you build a product on it, and do not repeat the "Apache" line that has been going around. At 2.4 trillion total parameters with 95 billion active it is a data center model regardless.

The split that works: run the 27B at home for anything private, offline, or repetitive, and reach the A95B through a hosted API when a task genuinely needs that scale. It is on DeepInfra, and both it and the 27B class of models sit in LU Labs Cloud if you want them next to each other in one picker.

FAQ

Is 27B enough? For local work it is the interesting size: big enough for real coding and agent loops, small enough to sit on one consumer card. The hybrid attention means long context does not price you out the way it does on a dense model of the same parameter count.

Ollama or LM Studio? Both pick up the shipped template automatically, which spares you the trap above. The bare-GGUF-plus-llama.cpp route is where people get bitten.

Uncensored builds? Apache 2.0 makes finetunes and abliterations fully legal, and community variants started appearing within a day of the weights. The stock model carries standard alignment.

How do I check the sizes myself? https://huggingface.co/api/models/<repo>/tree/main returns every file with its byte count. That is where every number in this post came from.

Run Ling 3.0 Flash Locally: 124B of Knowledge on a 96 GB Machine

David — Sun, 09 Aug 2026 12:56:06 +0000

The two big open releases on everyone's feed right now are Kimi K3 (2.8 trillion parameters, the first open 3T-class model) and Ling 3.0 Flash from Ant Group's inclusionAI. Only one of them can live on hardware a person owns, and it is not the one with the bigger headline. Here is the practical picture for running Ling 3.0 Flash locally, with measured file sizes instead of projections, plus the honest math on why K3 stays in the cloud.

The shape of the model decides everything

Ling 3.0 Flash is a 124B parameter Mixture of Experts that activates only 5.1B parameters per token (8 of 512 routed experts plus one shared). As with every MoE, total parameters set your memory bill and active parameters set your speed. 124B of knowledge at the memory counter, 5.1B of compute at the speed counter: that combination is why this model runs interactively on machines that would crawl under a dense 70B.

Two more architecture notes that matter in practice. It uses hybrid linear attention (35 Kimi Delta Attention layers alternating with 7 gated MLA layers), so long contexts grow memory gently instead of quadratically. And it is a native hybrid reasoner: it thinks before answering by default, and the thinking pass can be switched off per request, so you decide when to pay for reasoning.

The license is plain MIT. Weights are on Hugging Face under inclusionAI/Ling-3.0-flash, including official fp8, fp4 and int4 variants.

The numbers

Community GGUF conversions are up, and these sizes are measured from the repos, not estimated:

Quant	Size on disk	Realistic minimum
IQ1_S / IQ1_M	27 to 30 GB	36 GB RAM, visible quality cost
IQ2_M / Q2_K_XL	42 to 43 GB	48 to 64 GB
IQ3_XXS	51 GB	64 GB
Q4_K_M (sweet spot)	78 GB	96 GB
Q5_K_M	92 GB	128 GB
Q6_K	105 GB	128 GB
Q8_0	136 GB	192 GB

Realistic machine classes:

A Mac with 96 or 128 GB unified memory runs Q4_K_M comfortably, and 64 GB machines run the 2-bit and small 3-bit quants whole.
A 24 GB GPU plus 64 to 96 GB of DDR5 works well with MoE offload: attention layers and the shared expert on the GPU, routed experts in system RAM. Because only 5.1B parameters fire per token, generation speed stays in usable double digits.
32 GB or less: skip it and run a model that actually fits. The IQ1 files exist, but you will not enjoy them.

With a current llama.cpp:

llama-server -m Ling-3.0-flash-Q4_K_M.gguf \
  --ctx-size 32768 \
  --n-gpu-layers 99 \
  --n-cpu-moe 40

Start with a modest context window; the model was trained out to 256K, but every token of window is memory you could spend on a better quant instead.

The no-terminal route

If you would rather click than type flags, Locally Uncensored (open source, AGPL) wraps the whole thing: install, open the Model Manager, paste a Ling 3.0 Flash GGUF repo, pick the quant that fits your memory, chat. It handles the llama.cpp engine, the offload split, and keeps everything on your machine with no account and no telemetry.

And the Kimi K3 reality check

K3 deserves its headlines: first open 3T-class release, 1M token context, native image input, and it shares the Kimi Delta Attention lineage with Ling. But the smallest usable GGUF conversion, a 1-bit quant, is 466 GB on disk. The 2-bit is 861 GB. Q4 is 1.5 TB. Expert-pruned community builds squeeze toward the 512 GB server class, which is heroic and still not a gaming PC.

So the honest split for the price of one search query: run Ling 3.0 Flash at home, and reach K3 through a hosted API when a task genuinely needs the 1M window or vision. DeepInfra serves both (Ling at $0.03 in / $0.07 out per million tokens, which rounds to free; K3 at $2.85 / $14.25), OpenRouter lists a free Ling tier, and both landed in LU Labs Cloud on every plan this week, with reasoning as a toggle in both cases.

FAQ

Is it actually good? inclusionAI benchmarks it at parity with their previous 1T-class flagship on SWE-Bench Pro, agentic tool suites and long-context tasks. Vendor numbers, as always, but the architecture math is real, and the price of testing that claim yourself is a weekend download.

How fast is it locally? Speed tracks the 5.1B active count, not the 124B total. On Apple Silicon at Q4 and on GPU-plus-RAM hybrids, expect double-digit tokens per second.

Uncensored builds? None yet. The MIT license makes finetunes and abliterations fully legal, and a 5.1B-active model is cheap to tune, so expect community variants. The stock model carries standard alignment.

How to Run Qwen 3.8 Locally: The 2.4T Max Math, the Open Weight 27B, and What Runs Today

David — Mon, 03 Aug 2026 15:43:05 +0000

Alibaba made Qwen 3.8 Max generally available today (August 3): roughly 2.4 trillion parameters as a Mixture of Experts, a 1 million token context window, image and video input, and launch scores like 92.6 on GPQA Diamond and 86.6 on Terminal-Bench 2.1. The part that matters for this post: the open weight checkpoint is promised for the coming week, and it brings a consumer sized Qwen 3.8 27B along with it. Here is the practical picture for anyone who wants Qwen 3.8 on their own hardware.

The shape of the model decides everything

MoE models split the hardware question in two. Total parameters (2.4T) set your memory bill: every expert has to live somewhere. Active parameters (~95B per launch coverage) set your speed. That is why the Max serves cheaply in a datacenter, and also why it will never fit in your tower: all 2.4T parameters must sit in memory, because you never know which experts the next token activates.

The numbers for the Max

FP8, the native serving precision: ~2.4 TB of weights.
Q4, the local standard: ~1.2 TB.
An extreme 2 bit quant: ~600 GB, with real quality loss.

The largest single machine a consumer can buy in 2026 holds 512 GB of unified memory, less than half of the Q4 weights, before you allocate a single byte of KV cache for that million token context. Someone will chain Mac Studios together for a single digit tokens per second demo within weeks of the weights dropping. It will be a great video and a bad daily driver.

The 27B is the actual local story

This is the difference between this launch and Kimi K3 or DeepSeek V4 Pro: the same drop includes an open weight 27B. Projected from the Qwen 3.6 27B precedent:

VRAM	Quant class	Precedent size
8 GB	2 bit (UD-IQ2 class)	8.7 GB, reduced quality
12 GB	Q3_K_M	13 GB, RTX 3060 sweet spot
16 GB	Q4_K_M, recommended	~16 GB
24 GB	Q6_K, near lossless	~21 GB

Community GGUFs of open Qwen releases usually appear within days of the weights. If the 27B inherits even part of the Max's agentic gains (FrontierSWE jumped from 40.7 to 73.5 this generation), it becomes the default local model in its class more or less immediately. And weights mean derivatives: every open Qwen generation has received abliterated and heretic builds within weeks.

What to run today, not next week

Until the drop, the newest Qwen you can actually run is Qwen 3.6, and it is genuinely strong: the 27B dense runs from 8 GB VRAM up, the 35B MoE (3B active) is the coding pick at 24 GB. One line with a current llama.cpp era stack:

ollama pull qwen3.6:27b

If you would rather click than type, the free and open source local AI studio Locally Uncensored has the Qwen family in its one click model catalog, checks your memory before you download, and speaks to 12 local backends. The full setup walkthrough with quant tables lives here: How to Run Qwen 3.6 Locally, and the complete Qwen 3.8 hardware math is in Can You Run Qwen 3.8 Locally?

Why bother running it at home

Privacy: your prompts never leave the machine. Cost: open weights turn a metered bill into a one time hardware decision. Control: the model you benchmark today is the model you run next year, no silent upstream swaps. For the first time, the frontier launch everyone is hyping comes with a version of itself you will actually own, in the same week's drop.

DeepSeek V4 Flash vs V4 Pro: A Developer Decision Guide

David — Fri, 31 Jul 2026 17:01:42 +0000

With today's V4 Flash 0731 release, DeepSeek's V4 family now has two very different members, and picking between them is a real decision. Short version: Flash for agents and code, Pro for the hardest reasoning, Flash by forfeit if you want to run it yourself.

The spec sheet

	V4 Flash (0731)	V4 Pro (Preview)
Total params	284B	1.6T
Active per token	13B	49B
Context	1M in / 384K out	1M in / 384K out
API price per M	$0.14 in / $0.28 out	$0.435 in / $0.87 out
Responses API / Codex	Yes, native	No
Open weights	MIT	MIT
4 bit GGUF	~155 GB	~900 GB

The plot twist

Until this morning the story was: Pro is the smart one, Flash is the cheap one. The 0731 re post training broke that. Flash now posts 82.7 on Terminal Bench 2.1 and 54.4 on DeepSWE, and beats the V4 Pro Preview on several code and agent benchmarks while activating a quarter of the parameters. Pro keeps the lead where breadth and depth dominate: hard math, long proofs, dense multi document analysis over the full context window.

Cost wise the sticker gap says 3.1x on output, but agent reality widens it: agent loops burn output tokens (Flash's cheap side) and resend context (near free on cache hits for both). A day of heavy agent work that costs single digit dollars on Flash lands in the tens on Pro.

The local angle

This one is not close. Flash fits in about 155 GB at 4 bit and runs on a 192 GB Mac Studio or a big RAM workstation at interactive speeds, because only 13B parameters fire per token. Pro at 4 bit approaches 900 GB, which is multi GPU server territory and stays rented for almost everyone. If owning your model matters, the family decision is already made.

Picks by workload

Coding agents, terminal work, repo tasks: Flash 0731. It benchmarks ahead and speaks the Responses API that Codex style tools expect.
Everyday chat and writing: Flash. The quality gap does not justify 3x.
Hard math and research grade reasoning: Pro.
Full 1M context document analysis: Pro, hosted.
Your own hardware: Flash, by forfeit.

The cheapest way to run the comparison yourself is a flat plan where both sit in one model picker, for example LU Labs Hosted Pro, so switching per task is free and you notice which model you stop switching away from. The full head to head with sources is here: DeepSeek V4 Flash vs V4 Pro, the Pro specific options (including the self hosting math) here: How to Run DeepSeek V4 Pro, and the local setup for Flash here: How to Run DeepSeek V4 Flash Locally.

DeepSeek V4 Flash 0731: The Agent Benchmarks Behind Today's Hype

David — Fri, 31 Jul 2026 17:01:02 +0000

DeepSeek released the official build of V4 Flash today, named DeepSeek-V4-Flash-0731, and the interesting part is not that a new model dropped. It is what post training alone did to agent performance.

Same model, new job

The 0731 build keeps the exact architecture and size of the April preview: 284B total parameters, 13B active, 1M token context. DeepSeek only re ran post training, aimed squarely at agent workloads. The deltas:

Benchmark	Preview	0731
Terminal Bench 2.1	61.8	82.7
DeepSWE	7.3	54.4
NL2Repo	n/a	54.2

DeepSWE going from 7.3 to 54.4 without touching the base model is the kind of jump that usually needs a new generation. On several code and agent benchmarks the small Flash now outscores DeepSeek's own V4 Pro Preview, a 1.6T model activating nearly four times the parameters per token.

What that means for people building agents

Two shipping details matter as much as the scores. First, 0731 natively supports the Responses API format and is adapted for Codex, so agent tooling built on that interface runs against deepseek-v4-flash without a compatibility shim. Second, the pricing: $0.14 per million input tokens, $0.28 per million output, and $0.0028 on cache hits. Agent loops resend context constantly, which makes the cache hit price the number that decides your bill.

And the open part is not marketing: the weights landed on Hugging Face under MIT the same day, community GGUFs followed within hours, and the 155 GB 4 bit quant fits a 192 GB workstation or Mac Studio. A deep dive into what shipped, including the fine print (announced 2x peak hour pricing, the custom chat template, and where the model is unremarkable), is here: DeepSeek V4 Flash 0731 Explained.

Trying it without an API console

If you want to poke the model before wiring it into anything, the routes are: the official API (OpenAI compatible), OpenRouter (one model string), a flat price hosted plan such as LU Labs where V4 Flash and V4 Pro sit in one picker with monthly quota instead of per token billing, or fully local on your own hardware since the weights are open. The whole decision tree with working code samples is in the cloud guide, and the local path is covered in the local guide.

The strategic takeaway: specialization beat scale today. A 13B active model, trained hard on the agent loop, is outperforming a 49B active generalist on the workloads developers are building right now. Expect every lab to copy this recipe.

Run DeepSeek V4 Flash 0731 on Your Own Hardware: What It Takes

David — Fri, 31 Jul 2026 17:00:57 +0000

DeepSeek shipped the official V4 Flash build today (July 31), and it is the rare frontier release that real hardware can hold: 284B parameters as a Mixture of Experts, but only 13B active per token, MIT licensed, with GGUF quantizations on Hugging Face within hours. Here is the practical picture for anyone who wants it running locally this weekend.

The shape of the model decides everything

MoE models split the hardware question in two. Total parameters (284B) set your memory bill: every expert has to live somewhere. Active parameters (13B) set your speed: each token only touches a small slice, so the compute load resembles a 13B model. That is why V4 Flash runs interactively on machines that would crawl with a dense model of the same size.

The numbers

The unsloth GGUFs of the 0731 build come in two sizes: UD-Q4_K_XL at 155 GB and UD-Q8_K_XL at 162 GB. The 8 bit file is only 7 GB bigger because expert weights dominate and quantize well, so if you clear the bar for Q4 you should check whether Q8 fits too.

Realistic machine classes:

A Mac Studio with 192 GB unified memory runs Q4 comfortably.
A workstation with 192 to 256 GB DDR5 plus a 24 GB GPU works well: attention and shared layers on the GPU, experts in RAM. Expect roughly 5 to 15 tokens per second depending on memory bandwidth.
Anything smaller: take the Qwen 3.5 9B distill of V4 Flash, a 5.3 GB file that carries much of the reasoning style and runs on a 6 GB GPU or plain CPU.

One line serves the full model with a current llama.cpp:

llama serve -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL

The no terminal route

If you would rather click than type, the free and open source local AI studio Locally Uncensored has the V4 Flash family in its one click model catalog (the full build, the popular abliterated variant, and the 9B distill), checks your memory before you download 150 GB, and speaks to 12 local backends including llama.cpp.

The full walkthrough with all download links lives here: How to Run DeepSeek V4 Flash Locally, and the complete memory math per machine class is in Can You Run DeepSeek V4 Flash Locally?

Why bother running it at home

Three reasons. Privacy: your prompts never leave the machine. Cost: the weights are MIT, so a one time hardware investment replaces a metered bill. And control: the model you benchmark today is the model you run next year, no silent upstream swaps. With agent scores like 82.7 on Terminal Bench 2.1, this is the first time the model everyone is hyping on release day is also one you can own.

Abliterated Models Guide - Qwen 3.6, Gemma 4 Heretic, Llama 3.1 Uncensored Download Links

David — Fri, 24 Apr 2026 13:58:02 +0000

Originally published on locallyuncensored.com

If you've looked at the Discover tab in any local-AI app and wondered why some Llama variants have abliterated in the name, this is the post that explains it. Plus the curated download list for 2026.

What Abliteration Actually Is

Modern instruction-tuned LLMs have a learned refusal direction in their residual stream. When a prompt activates that direction strongly enough, the model outputs "I cannot help with that." The direction was put there during RLHF.

Abliteration removes it via orthogonalisation. You take a corpus of refused prompts, isolate the activation pattern that distinguishes them from accepted prompts, then project that direction out of every weight matrix. The result is a model with the same training and capabilities but no longer prone to categorical refusal.

It's a clean technique - not a finetune, not a jailbreak, not a system-prompt trick. Original paper: "Refusal in Language Models Is Mediated by a Single Direction" (Arditi et al., 2024).

Abliterated vs Other Uncensored Approaches

Method	How it works	Effort	Quality impact
Abliteration	Project out refusal direction	hours on GPU	1-3% degradation
Full finetune (Dolphin, Hermes)	Re-train on uncensored corpus	days, expensive	Variable
LoRA finetune	Adapter on uncensored data	hours	Minor, reversible
Merge (Frankenmerges)	Combine multiple finetunes	hours	Highly variable
System prompt jailbreak	Persona-style instructions	None	Brittle

Abliteration is the cleanest research-grounded option. Dolphin and Hermes are battle-tested production finetunes.

Recommended Abliterated Models (2026)

Qwen 3.6 Family

richardyoung/qwen3-14b-abliterated:q4_K_M - 9 GB, fits 12 GB VRAM, vision-capable. Comes in :q4_K_M (chat) and :agent (tool-calling) tags via Ollama.
Qwen 3.6 27B Samantha (huihui-ai variant) - abliterated dense 27B with the Samantha personality finetune.

Gemma 4 Heretic

Stabhappy/gemma-4-31B-it-heretic-Gguf - Gemma 4 31B base abliterated. ~17 GB at Q4_K_M. Native vision, tool calling.
Gemma 4 26B MoE HERETIC - 26B brain with 4B active. Smaller VRAM peak, MoE-fast inference.

Llama 3.1 Family

mannix/llama3.1-8b-abliterated:q5_K_M - 5.7 GB. The most-pulled abliterated Llama on Ollama. Comes with :agent tag for tool calling.
mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated - the canonical reference variant.

Hermes 3

Hermes 3 is technically a full finetune, not abliteration, but functions similarly:

hermes3:8b via Ollama - 4.7 GB, fits 8 GB GPUs. Good chat default.
hermes3:70b - 40 GB, needs 48 GB VRAM or aggressive quantisation.

GLM 5.1 Heretic

The newest entrant: huihui-ai/Huihui-GLM-5.1-abliterated-GGUF. The 754B MoE GLM 5.1 abliterated. 236 GB at IQ2_M - not consumer hardware, but if you have a Mac Studio M4 Ultra, it's the strongest open abliterated model period.

How to Download and Run

Path 1 - Ollama (one command)

ollama pull richardyoung/qwen3-14b-abliterated:q4_K_M
ollama run richardyoung/qwen3-14b-abliterated:q4_K_M

Path 2 - Locally Uncensored (one click)

Open Locally Uncensored, navigate to Model Manager > Discover > Text, click the UNCENSORED filter tab. The 34 curated abliterated GGUFs are all there with one-click download.

The new v2.4.0 Settings > Model Storage override lets you redirect the GGUF download folder if you want them on a separate drive.

Hardware Recommendations

VRAM	Best Abliterated Pick	Why
8 GB	Llama 3.1 8B abliterated Q4_K_M	Fits with headroom
12 GB (RTX 3060)	Qwen 3 14B abliterated Q4_K_M	Sweet spot, ~15 tok/s
16 GB	Gemma 4 31B Heretic Q4_K_M	Best general-purpose at this VRAM
24 GB (RTX 3090/4090)	Gemma 4 31B Heretic Q5_K_M	Higher quality
48 GB+	Hermes 3 70B or GLM 5.1 Heretic IQ2	Frontier-tier quality

Common Questions

Will an abliterated model write me malware?

Probably not the way you're thinking. Abliteration removes the categorical refusal but the model still has training-time priors against obviously-bad outputs. The models work best for legitimate-but-edge-case use cases: security research, fiction with violence, medical questions the base model deflects, legal grey areas, adult creative writing.

Are abliterated models dangerous?

No more than the underlying base. Abliteration removes a layer of guardrails. The model's underlying knowledge is unchanged from the base.

Can I abliterate a model myself?

Yes. The technique is well-documented and the code is on GitHub (search abliterator). You need a GPU with the model loaded, a few thousand refused-vs-accepted prompt pairs, and a few hours.

Locally Uncensored is AGPL-3.0 licensed. Built by PurpleDoubleD. Bug reports on GitHub Discussions or in the Discord.

How to Run Qwen 3.6 Locally - 27B Dense, 35B MoE, and Coding Variants Setup Guide

David — Fri, 24 Apr 2026 13:58:02 +0000

Originally published on locallyuncensored.com

Qwen 3.6 dropped on April 21 2026. Two main families: a 27B dense model that activates every parameter per token and a 35B MoE with 3B active per token. Both ship with vision, agentic coding, thinking-mode preservation, and a 256K context window.

If you only have time for the short version: install Locally Uncensored, open Model Manager > Discover > Text, search Qwen 3.6, hit the download arrow on the variant that fits your VRAM.

Which Qwen 3.6 Variant Should You Pick?

The biggest decision is dense vs MoE. The second biggest is which quant.

The 27B dense activates all 27B parameters for every token. Slower per token, but every token gets the full model. Quality is consistent. Recommended default for general chat, reasoning, and most coding.

The 35B MoE only activates 3B parameters per token via routing. Much faster per token (often 2-3x throughput at similar quants). VRAM peak during inference is lower than the model size suggests. But routing introduces variance. The MoE wins on coding benchmarks (SWE-bench specifically) when you pick the coding-specialised variant.

Quant Comparison Table

Variant	Quant	Disk	VRAM Target	Quality
27B dense	UD-IQ2_XXS	8.7 GB	8 GB GPU	Good (low-VRAM lifesaver)
27B dense	Q3_K_M	13 GB	12 GB GPU	Very good (RTX 3060 sweet spot)
27B dense	Q4_K_M	16 GB	16 GB GPU	Recommended default
27B dense	UD-Q4_K_XL	16 GB	16 GB GPU	Better quality per GB
27B dense	Q5_K_M	18 GB	20 GB GPU	High
27B dense	Q6_K	21 GB	24 GB GPU	Near-lossless
27B dense	Q8_0	27 GB	32 GB GPU	Effectively lossless
35B MoE	Q4_K_M	24 GB	24 GB GPU	Recommended for MoE
35B MoE	NVFP4	22 GB	22 GB GPU (RTX 40+)	Smallest with full quality on Blackwell
35B MoE coding	NVFP4	22 GB	22 GB GPU (RTX 40+)	Best coding-bench-per-GB
35B MoE	BF16	71 GB	96 GB GPU	Reference quality

Recommendation by Hardware

8 GB VRAM (RTX 3060 8GB, RTX 4060 8GB): 27B UD-IQ2_XXS - the only quant that fits
12 GB VRAM (RTX 3060 12GB, RTX 3080 Ti, RTX 4070): 27B Q3_K_M - sweet spot, ~15-25 tok/s
16 GB VRAM: 27B Q4_K_M or UD-Q4_K_XL - the recommended default
24 GB VRAM (RTX 3090, RTX 4090): 27B Q6_K for max dense quality, OR 35B MoE Q4_K_M for coding
RTX 40+ Blackwell: 35B MoE NVFP4 - smallest size with native quality
Apple Silicon M3/M4: 35B MoE MLX BF16 via MLX runtime
CPU only with 32 GB RAM: 27B Q4_K_M at 1-3 tok/s - usable for short tasks

Installation Path 1 - Ollama (CLI)

ollama pull qwen3.6:27b           # dense Q4_K_M, 16 GB
ollama pull qwen3.6                # 35B MoE Q4_K_M, 24 GB
ollama pull qwen3.6:35b-a3b-coding-nvfp4   # coding NVFP4
ollama run qwen3.6:27b

Installation Path 2 - Locally Uncensored (GUI)

If you want a one-click experience plus chat, agent mode, image generation, and a/b model compare in the same window:

Download the v2.4.0 installer for your OS
First-launch wizard auto-detects Ollama (or offers one-click install)
Model Manager > Discover > Text > search Qwen 3.6
Click the download arrow on the variant matching your VRAM

Performance on RTX 3060 12 GB

Tested with Qwen 3.6 27B Q3_K_M, 4096-token context, fp16 KV cache:

Workload	tok/s
Cold first response	~3 (model load)
Warm chat (50-token answers)	22-26
Long-form (1000 tokens)	18-20
Thinking-mode enabled	15-18

Vision Support

Both 27B dense and 35B MoE accept image input. Drag-and-drop a screenshot, photo, or chart. VRAM cost for vision is +1-2 GB on top of the base model.

Coding Performance

The 35B MoE coding-specialised variants are tuned on SWE-bench training data. The coding NVFP4 variant scores in the same ballpark as Claude 3.5 Sonnet on SWE-bench-verified at a fraction of the inference cost.

For day-to-day coding inside LU's Codex agent, the 27B dense Q4_K_M is the better default - consistent quality, no MoE-routing variance.

Qwen 3.6 vs Qwen 3.5

Feature	Qwen 3.5	Qwen 3.6
Vision	No	Yes (both 27B and 35B)
Context window	128K	256K
Thinking mode	QwQ-only	Preserved across variants
Coding-specific MoE	No	Yes (35B-a3b-coding)
NVFP4 quant	No	Yes (35B MoE)
MLX variant for Apple Silicon	No	Yes

Locally Uncensored is AGPL-3.0 licensed. Built by PurpleDoubleD. Bug reports on GitHub Discussions or in the Discord.

Locally Uncensored v2.4.0 — Settings Polish, Linux Drag Fix, and Configurable HuggingFace Path

David — Fri, 24 Apr 2026 13:53:15 +0000

Originally published on locallyuncensored.com

Locally Uncensored v2.4.0 is a polish release. Eight fixes, two of them surfaced through community feedback on Discord, six caught during an internal end-to-end pass on the v2.3.9 build. No new headline features — this release exists so the next feature release lands on a cleaner foundation.

TL;DR

Single-instance lock — double-clicking the shortcut focuses the existing window instead of spawning a second process
Settings → Model Storage — paste or pick the folder where HuggingFace GGUF downloads land
Settings → Privacy — in-app statement of what runs locally and what doesn't
Settings → Onboarding — a button that re-runs the first-launch wizard on demand
Reset tutorial — the button now actually does what its label promises
Linux window drag — the title-bar drag works on Ubuntu 24.04 again
Discover — the HuggingFace download path is no longer printed twice
HuggingFace search heuristic — search results for repos with a quant tag in the name no longer 404 on download

Single-Instance Lock

Before v2.4.0, double-clicking the desktop shortcut started a second locally-uncensored.exe process. Both instances would race each other writing to the store backup file — not a frequent corruption source, but a real one when both happened to flush at the same millisecond.

v2.4.0 ships with tauri-plugin-single-instance. The second launch focuses, un-minimizes, and brings the existing window to front. No new process. Verified with three back-to-back launches: only one PID survives.

Settings → Model Storage — Configurable HuggingFace Folder

The Model Manager → Discover → Text tab lets you download GGUF models from HuggingFace. Until v2.4.0, the destination folder was always auto-detected from the active openai-compat provider — usually LM Studio's models folder.

That worked fine for single-disk setups. It did not work for dual-boot users who wanted a shared model partition between Linux and Windows, or anyone running a NAS-mounted models folder. Reported on Discord by diimmortalis.

v2.4.0 adds a dedicated Settings → Model Storage section with a path input, a Browse button, and a Reset button. The override takes effect immediately. Verified end-to-end with a Gemma 4 E4B download (4.6 GB) landing in a custom folder while the LM Studio default folder stayed untouched.

Linux Window Drag Fix

On Ubuntu 24.04 the title-bar drag threw an unhandled Promise rejection — core:window:allow-start-dragging was missing from the capability list. Reported on Discord by diimmortalis with a clean Promise-rejection dump. One-line fix in src-tauri/capabilities/default.json.

Tests & Verification

Test suite went from 2205 to 2216 (+11 regression tests). cargo check clean. tsc --noEmit clean.

Live end-to-end on the installed v2.4.0 build:

Single-instance: 3 back-to-back exe launches → 1 PID
HF download override: typed custom path, Discover subtitle updated, Gemma 4 E4B partial download (35.9 MB at 897 KB/s) landed in the picked folder
Re-run onboarding: click → marker deleted → 6-step wizard renders → marker re-created
Reset tutorial: click → flag flipped → new chat → Agent toggle → tutorial renders

Download

GitHub Releases. Signed installers for Windows (.exe, .msi) and Linux (.deb, .rpm, .AppImage). Auto-update picks the new build up on next launch for anyone on v2.3.x.

Locally Uncensored is AGPL-3.0 licensed. Built by PurpleDoubleD. Bug reports on GitHub Discussions or in the Discord.

Anthropic is Rationing Claude Code on Pro — Here's a Local Alternative

David — Thu, 23 Apr 2026 10:34:18 +0000

Earlier this week, Anthropic ran a quiet test: a small slice (~2%) of new Pro plan subscribers found that Claude Code wasn't included with their $20/month subscription. The pricing page was updated to reflect this. It made some noise on Reddit and X, Anthropic walked it back, and the page was reverted.

But the incident highlights something real: the economics of hosted AI are strained.

What Actually Happened

Anthropic's head of growth clarified on social media that the test affected about 2% of new prosumer signups. The reasoning was straightforward: usage patterns have changed dramatically. Users have moved from brief chat sessions to "nearly always-on, multi-agent workflows" that consume vastly more tokens. The current plans weren't built for this.

To be clear: this wasn't a crisis. It was a business experiment that got rolled back quickly. But it was also a signal — one that shouldn't be surprising if you've been paying attention to how compute-heavy AI tools have become.

The Trend Is Clear

Claude Code isn't unique here. OpenAI has introduced peak-hour caps. Anthropic has added tighter limits during high-traffic periods. Gemini, ChatGPT, and others have all introduced various forms of rate limiting as agentic workflows (long-running, multi-step tasks) have taken off.

This isn't malice — it's math. Running a model that can handle complex, hours-long agentic tasks requires significant GPU compute. At $20/month, there's a real gap between what heavy users consume and what the subscription covers.

Enter Local Models

This is where running AI locally becomes genuinely compelling, not just theoretically interesting.

Tools like Ollama, LM Studio, and Locally Uncensored let you run capable language models on your own hardware. No subscription. No per-token billing. No rate limits. No plan changes.

The tradeoff is real: you need decent hardware (a modern Mac with unified memory, a gaming PC with a good GPU, or a dedicated home server), and the experience differs from hosted APIs. But for developers who rely on agentic workflows — the exact users feeling the squeeze from providers — the local path is increasingly viable.

Recent open-weight models from Mistral, Qwen, and the Llama family are genuinely capable for coding tasks. They're not matching the frontier models on every benchmark, but for the majority of real-world dev work, the gap has shrunk considerably.

Is This a Sales Pitch?

Not really — and I want to be clear about that. If Anthropic's pricing works for you and you don't hit limits, there's no urgent reason to change. Their models are excellent.

But if you've been on the receiving end of a rate limit mid-flow, or if you're watching your usage climb and wondering what happens next, it's worth knowing that the local option exists and has gotten significantly easier to set up over the past year.

The local ecosystem isn't for everyone. But for developers who have built automated workflows around AI — the exact users Anthropic was quietly trying to ration — it might be worth an afternoon of experimentation.

What do you think — is the local-first approach realistic for your use case, or are you all-in on hosted APIs? I'd genuinely like to know what you're running into.

qwen3.6-27b scores 77.2% on SWE-bench. the dense model is winning against MoE.

David — Thu, 23 Apr 2026 07:30:20 +0000

When Alibaba released Qwen3.6-35B-A3B, the MoE (Mixture of Experts) design stole all the headlines. 35 billion parameters, 3 billion activated per token — everyone's been focused on that ratio.

Then they dropped Qwen3.6-27B. A plain old dense model. 27 billion parameters, all active.

On SWE-bench Verified, the 27B dense scores 77.2%. The 35B MoE scores 73.4%. The dense model is outperforming the MoE by nearly 4 points — on the benchmark that measures real software engineering capability.

what SWE-bench actually measures

SWE-bench gives an LLM a real GitHub issue and a codebase. It has to understand the problem, find the right files, write the fix, and get the tests to pass. It's not multiple choice — it requires actual coding.

Qwen3.6-27B at 77.2% puts it in range of proprietary models. Claude Opus 4.5 scores 80.9%. The gap is real but narrowing — and Qwen3.6-27B does it on your own GPU under Apache 2.0.

why is the dense model winning?

Two factors seem to be driving this:

1. Full parameter utilization. In a MoE model like the 35B-A3B, only 3B of 35B parameters are active per token. The routing layer decides which experts to use. This is efficient for inference speed, but the model can't "use" all of its knowledge simultaneously. A dense model can activate its full capacity for harder reasoning tasks.

2. Architecture: Gated DeltaNet. Qwen3.6-27B isn't a vanilla dense transformer. It uses a Gated DeltaNet + Gated Attention hybrid — alternating layers of linear-gated attention (DeltaNet) with standard gated attention. DeltaNet processes information in compressed deltas rather than full representations, which lets it handle long contexts more efficiently while maintaining reasoning depth.

The result is a model that can do 262K context natively (extendable to 1M tokens) while still being a strong coder.

the benchmark breakdown

Task	Qwen3.6-27B (dense)	Qwen3.6-35B-A3B (MoE)	Gap
SWE-bench Verified	77.2	73.4	+3.8
SWE-bench Pro	53.5	49.5	+4.0
Terminal-Bench 2.0	59.3	51.5	+7.8
SkillsBench Avg5	48.2	28.7	+19.5
QwenWebBench	1487	1397	+90
NL2Repo	36.2	29.4	+6.8

Terminal-Bench (real terminal operations) and SkillsBench show the largest gaps. These are tasks where the model needs to chain together multiple operations — the kind of thing where full parameter access seems to matter most.

the tradeoff

Dense models aren't free. The 27B activates all 27B parameters per forward pass. The 35B MoE activates only 3B. During inference:

35B MoE is faster per token (3B vs 27B compute)
35B MoE uses less memory for the active computation (but total disk/loaded size is still large)
27B dense is better at hard coding tasks (SWE-bench, terminal operations)

If you're doing simple chat, the MoE will be faster. If you're running an agent that needs to reason through a complex codebase — the dense model is showing real advantages.

vision included

Qwen3.6-27B is an image-text-to-text model. The vision encoder is built in. That means you can screenshot a UI and ask it to fix the bug, read a diagram and explain the architecture, or debug from screenshots. The 35B MoE is text-only.

running it

ollama run qwen3.6-27b

With Locally Uncensored, you also get image input, a built-in code agent, and fully local outputs:

git clone https://github.com/PurpleDoubleD/locally-uncensored
cd locally-uncensored && npm run tauri dev

The MoE vs dense debate isn't settled. But on coding agent tasks, Qwen3.6-27B is making a strong case that raw parameter count isn't everything — architecture and full utilization matter too.

Locally Uncensored — AGPL-3.0 license.