DEV Community: David

How to Run Qwen-Image 2.1 Locally: One Model That Draws and Edits

David — Tue, 22 Sep 2026 10:13:53 +0000

Alibaba Qwen published Qwen-Image 2.1 on 20 September 2026. The interesting part for anyone running models on their own hardware is not the benchmark table, it is the file count: one set of weights now does text to image and image editing. There is no separate edit model in this generation.

The less interesting part, which you should read before you spend 16 GB of bandwidth, is the license. More on that at the end, and it is not a footnote.

What is actually in the box

The architecture moved, and it moved downwards:

Image model: 7B parameters, single stream DiT, 32 layers
Text encoder: Qwen3-VL-8B
VAE: new, 64 channels, RGBA
Objective: flow matching

For comparison, the original Qwen-Image from August 2025 (the one usually written as 2508) had a 20B image model and a Qwen2.5-VL encoder. So 2.1 is not a bigger sibling of the old model, it is a different and smaller one. Anything you tuned against the 20B shape has nothing to attach to.

New in 2.1 against everything before it: a native alpha channel, up to ten reference images in one pass, local editing driven by a drawn marking rather than only a painted mask, a prefix KV cache to speed up repeated edits, and better typography. Native 2K carried over from 2.0 in February 2026.

Memory

I have not benchmarked this model, so these are quoted numbers, not mine. ai.rs puts the int8 pipeline at roughly 16 GiB in total: about 6.8 GiB for the image model, 8.7 GiB for the text encoder and 0.6 GiB for the VAE. A 24 GB card keeps all three parts resident. On a 16 GB card the usual approach is to push the text encoder into system RAM and let the card carry the rest. The full bf16 weights are a different machine entirely at 33 to 40 GB.

ai.rs also reports 5.93 seconds on an RTX 5090. Their hardware, their measurement, quoted as such.

The three files

The int8 repack from Comfy-Org is three files that go into three different ComfyUI folders:

ComfyUI/models/diffusion_models/qwen_image_2.1_int8_convrot.safetensors
ComfyUI/models/text_encoders/qwen3vl_8b_int8_convrot.safetensors
ComfyUI/models/vae/qwen_image_2.1_vae_bf16.safetensors

Sizes, as Hugging Face lists them and as they measure on disk: 7.26 GB (6.76 GiB), 9.35 GB (8.71 GiB), 676 MB (0.63 GiB). Full bf16 versions of the first two exist at 14.2 GB and 17.5 GB, and there is a w4a8 encoder variant at 6.31 GB if you want to trade quality for memory.

The version gate you will hit first

ComfyUI added support in 0.37.0, merged on 19 September 2026, and it brought a new encode node:

TextEncodeQwenImage21

The older Qwen-Image edit nodes do not fit this model. If you are assembling a graph by hand and it fails at the encode step, check your ComfyUI version before you check anything else.

If you would rather not assemble the graph at all, Locally Uncensored 3.0.1 (AGPL-3.0, Windows and Linux) ships it as a Model Manager bundle called "Qwen-Image 2.1 (Generate and Edit)": 16.1 GB, three files, one click, into the folders above. It refuses politely on an older backend with exactly this line:

Qwen-Image 2.1 needs ComfyUI 0.37.0 or newer. Update ComfyUI in Settings.

Which is a small thing, but a better small thing than a graph that dies two nodes in.

Values that work

Straight from the official template:

Steps: 25
CFG: 1
Sampler: euler
Scheduler: simple
Resolution: 1024x1024

CFG below 1 is not usable with this model. If you are used to dialling CFG down on other models to soften things, that instinct will only produce noise here.

Generating and editing

Generation is what you expect: prompt in, negative prompt optional, picture out.

Editing is the part worth adjusting to. You give it one reference image and a prompt, and there is no mask. You describe the change in words instead of painting where it should happen. In Locally Uncensored the mask path is closed for this model on purpose, and the output follows the reference image while the resolution follows the canvas you picked. Whether you find that liberating or annoying probably depends on how much of your workflow currently involves a brush.

What is not wired up yet

In Locally Uncensored 3.0.1, several of the model's own abilities are not exposed. Stated flat, with nothing said about plans:

Several reference images at once. The model takes ten, the app takes one.
RGBA output. The VAE has the channel, the app does not write it out.
Masks with this model.
A prompt enhancer and the KV cache node.

And across the board, not app specific: there is no Lightning or Turbo LoRA for 2.1 as of 21 September 2026. The old Qwen-Image speed LoRAs were built for the 20B architecture and do not fit. If your habit is to reach for a 4 step LoRA, there is nothing to reach for yet.

The license

Qwen-Image 2.1 is published under the Qwen Research License. Research and evaluation are covered. Commercial use is not, and needs a separate license from Alibaba Qwen.

This is a real break with the rest of the series. Qwen-Image 2512 and Qwen-Image-Edit 2511 are Apache 2.0 and they stay Apache 2.0. So if you are generating assets for paid work, or building something you intend to sell, the newest model in the family is not the one to build on as published, and the older pair is.

Worth saying plainly because the naming does not warn you. It reads like the next version of an Apache 2.0 model, and the terms underneath are not the same terms.

Short version

A 7B model that draws and edits with one set of weights, about 16 GiB of int8 pipeline, ComfyUI 0.37.0 or newer, 25 steps at CFG 1, one reference image and no mask when editing, and a license that stops at research and evaluation. If that last clause does not block what you are doing, it is a cheap thing to try. If it does, the Apache 2.0 entries in the same series are still there.

The longer walkthrough with screenshots of each step lives on the Locally Uncensored blog.

DeepSeek V4.1 Flash is out: 552B, MIT, and nothing can run it yet

David — Thu, 10 Sep 2026 19:23:12 +0000

DeepSeek published DeepSeek-V4.1-Flash on Hugging Face on 10 September 2026 at 02:17 UTC, MIT licensed and ungated. I spent the day reading the files and poking the model through an OpenAI compatible endpoint instead of reading the announcement. Here is what is worth knowing before you start a download measured in hundreds of gigabytes.

TL;DR

552B backbone, 8B active in prefill and 16B in decode, 1,048,576 token context, reads images.
Original checkpoint: 510.3 GB across 48 shards, FP8 with FP4 routed experts.
Smallest community quant on day one: 168.9 GB. There is no four bit build yet.
No released runtime can execute any of it. llama.cpp support is pull request 28696, opened the same day, still a draft, and it is a convert PR.
The KV cache is 890 bytes per token. A full million token context is about 930 MB. That is the actual release.
The reasoning effort is a 1 to 100 dial, not a switch, and telling it to stop makes the turn longer.

The number that fools people

Eight billion active parameters in prefill, sixteen in decode. Both figures are real and both describe compute per token, not memory. The router picks six of 384 routed experts per token and picks differently for the next one, so all 552B have to stay reachable.

There is a second store on top: DeepSeek calls it Engram, a conditional memory of 196B parameters looked up sparsely per token. It is in the checkpoint and it is not small.

Active parameters decide how fast it runs. Resident weights decide whether it starts.

What the quants actually weigh

Measured from the Hub on the evening of the release, summed per build:

Build	Size	Notes
Original FP8 with FP4 experts	510.3 GB	48 shards, DeepSeek's own
MLX 2 bit with MTP	238.8 GB	Apple silicon
GGUF Q2_K	191.8 GB	7 shards
GGUF mixed IQ2_XXS + Q2_K	168.9 GB	smallest published

The smallest one averages 2.25 bits per weight across the routed experts, and its author says plainly that it was made without an importance matrix, on a constant unit importance vector, as a memory bounded choice rather than a calibrated one. Two bit without calibration is not a free lunch.

The wall

You can download 168.9 GB today. You cannot run it.

Support lives in llama.cpp PR 28696, convert : add DeepSeek V4.1 (DeepseekV41ForCausalLM), opened 10 September 2026 at 10:23 UTC, about eight hours after the weights, and still marked draft. Read the title: it teaches the converter to write the GGUF. Writing the file and executing it are separate pieces of work.

The person who published the mixed two bit build says the same thing on the model card, and it is the most useful sentence written about this model so far: it is a weights artifact, and no runtime has yet been shown to execute it end to end.

This is normal for day one. GLM-5.3-Flash sat behind an unmerged PR for the same reason two weeks ago. It resolves in days or weeks. It has not resolved yet.

One trap if you convert it yourself

DeepSeek V4.1 stores FP8 block scales in 32 by 32 blocks. DeepSeek V4 used 128 by 128.

Carry the old value across and every weight in the model is silently rescaled. No error, no warning. You get a file that loads, runs and produces confident nonsense, which is the worst failure mode there is because it looks like a bad quant rather than a bad conversion.

If you see token soup out of a self converted build, check the block size before you blame anything else.

The part that is actually clever

Long context has a cost most tables hide. Every token leaves a key and value entry behind that stays in memory for the life of the conversation. On an agent that reads a repo, runs a command, reads the output and repeats for two hundred steps, that cache is the bill.

Three things bring it down here:

A causal encoder decoder. 40 layers split into a 20 layer causal encoder and a 20 layer decoder. The decoder does not derive its global KV from each of its own layers, it projects it from the final encoder hidden states. One projection instead of twenty caches.
CSA2. Each attention layer gets one of three static modes, Full, Reindex or Reuse, so layers share main KV and indexer keys. A hierarchical sparse indexer in the decoder restricts later indexing layers to a candidate pool built by the first Full layer, which stops indexing cost growing with context length.
FP4 for the cache itself, E2M1 with one E4M3 scale per sixteen channels. Four bit weights are common. Four bit cache is not.

Result: 890 bytes per token, which DeepSeek puts at about a quarter of V4 Flash and one four hundred and thirty seventh of V1. A million token window costs roughly 930 MB of cache.

Their published benchmark table has exactly the shape you would predict. On agentic tests it takes the top row repeatedly against models with far more active compute: Terminal-Bench 2.1 at 90.6, DeepSWE v1.1 at 74.2, AutomationBench at 54.8, Codeforces rating 3471. On single shot reasoning it does not, and it lands slightly below its own V4 Pro sibling on GPQA Diamond. Those are vendor numbers, read them as such, but the pattern is consistent enough to be believable.

What the reasoning dial does when you tell it to stop

The model card describes a continuously controllable reasoning effort from 1 to 100. Every instruct benchmark was run at 100.

That collides with the OpenAI compatible convention, where reasoning_effort is a coarse setting with a none position. I ran the same question three times each way through DeepInfra, 700 max tokens, nothing else changed:

Run	No signal	`reasoning_effort: "none"`
1	42 out, 123 chars reasoning	82 out, 0 chars
2	42 out, 118 chars reasoning	53 out, 0 chars
3	42 out, 123 chars reasoning	51 out, 0 chars

The off signal did not stop the thinking. It stopped the thinking from being labelled, and the monologue moved into content, unmarked. In two of three runs no actual answer fit in the budget.

If you are building on this model: treat it as always reasoning, and do not offer your users a switch that makes their replies longer and dearer.

One more thing worth knowing. reasoning_effort: "max" is accepted, but acceptance proves nothing: an invalid value comes back 422 with the whole enum, which is provider level validation rather than a model capability. Measured on a hard prompt, max produced 308 characters of reasoning against 337 for high. There is no higher rung to reach for.

Two things I checked because tags lie

Vision. The card says multimodal, so I sent three 8x8 PNGs of different colours with one question each. Red, blue, green, correct every time. The older V4 Flash 0731 returns an error on image input, so this is genuinely new in the line.

Tools. A single get_weather function definition came back as a real tool_calls entry with {"city": "Berlin"}. Native, no prompt translation needed.

What to run locally instead this week

If you want a capable model on hardware you own today:

Qwen 3.8 27B, 10.7 GB to 54.7 GB depending on the quant, Apache 2.0. The one that fits a single card.
Ling 3.0 Flash, 27 GB at the smallest build, MIT, 5.1B active.
GLM-5.3 Flash, 93.1 GB at the smallest build, MIT, reads images.

And if you want V4.1 Flash itself before the runtime lands, it is a hosted model for now, the same way Qwen 3.8 was until its open checkpoint hit HuggingFace on 12 August.

I maintain Locally Uncensored, a desktop app for running these models on your own machine, and the hosted catalogue at lu-labs.ai picked this one up on release day. The measurements above were taken while deciding how to label it in our own model picker, which is why they exist at all.

How to run Qwen 3.8 without a GPU (and which Qwen 3.8 you can actually download)

David — Mon, 07 Sep 2026 21:13:29 +0000

Disclosure, since half of this post is about a service we run: LU Labs is the hosted layer next to our free open source desktop app for Windows and Linux. The other half is about running Qwen 3.8 on your own hardware, which costs us money rather than making it, and I have written that part first because for a lot of readers it is the right answer.

"How do I run Qwen 3.8 without a GPU" is really two questions wearing one coat, and they have opposite answers depending on which Qwen 3.8 you mean.

First, work out which model you are talking about

Alibaba shipped several things under the same version number in August 2026, and the licences are not the same either.

Qwen 3.8 27B is a dense 27 billion parameter model, Apache 2.0, and it runs on a single graphics card or a decent laptop. This is the one you can download.

Qwen 3.8 Max and Qwen 3.8 A95B are the flagships. A95B is 2.4 trillion parameters in total with 95 billion active per token, and all 2.4 trillion have to be resident while it works. That is a rack, not a desktop. The weights for A95B are public, but under Alibaba's own qwen3.8-max terms rather than Apache, so if you are checking licences for a commercial build, read them rather than assuming.

For the flagships, the choice is not local against hosted. It is hosted against not using the model.

If you meant the 27B: run it yourself

Genuinely, run it. At Q4_K_M the file is about 17 GB, which loads whole onto a 24 GB card with room left for context, spills into system RAM on a 16 GB card (slower, still works), and is comfortable on a 32 GB Apple Silicon Mac. IQ4_XS at about 15.7 GB is the largest quant that stays fully on a 16 GB card.

The one thing that trips almost everyone: Qwen 3.8 ships a chat template, and the template is not decoration. It marks where your message ends and the answer begins, and it carries the reasoning switch. Load a bare GGUF without it and you get one of two failure modes that both look like a bad quantization: it rambles and never stops, or it answers in a clipped voice and forgets the conversation between turns. In llama.cpp the fix is the --jinja flag. LM Studio and Ollama usually bring the template along with the model, so this mostly bites people who pulled a raw file by hand.

The longer version, with the full quant table and the vision file, is in how to run Qwen 3.8 27B on your own computer.

Everything below is about the flagships.

Reaching Max and A95B in a browser or from code

Both are in our chat catalog. Worth stating clearly, because our own older posts said otherwise: the catalog is now the same on every plan and on every credit pack. There is no shortlist, no model reserved for an expensive tier, and a €5 pack bought with no subscription reaches both flagships exactly like a €99 month does. A bigger plan buys credits, not access.

In the browser: sign in at lu-labs.ai, open the chat tab, and both appear under their catalog names, Qwen 3.8 Max and Qwen 3.8 A95B. Nothing downloads and there is no driver to install.

From code: Settings, then Cloud API keys, create one. It starts with lu_ and is shown once, since we keep a hash rather than the key. Base URL https://lu-labs.ai/api/inference/v1, OpenAI chat completions shape, so Aider, LibreChat, your own script or plain curl all work unchanged.

curl https://lu-labs.ai/api/inference/v1/chat/completions \
  -H "Authorization: Bearer $LU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3.8-Max",
       "messages":[{"role":"user","content":"Hello"}]}'

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://lu-labs.ai/api/inference/v1",
    api_key=os.environ["LU_API_KEY"],
)

resp = client.chat.completions.create(
    model="Qwen/Qwen3.8-2.4T-A95B",
    messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)

Both ids are exact catalog strings. Both take tool definitions in the standard tools parameter and call them natively, so agent scaffolding needs no adapter. Neither reads images: attach a screenshot and the request comes back rejected, which is a property of these two ids rather than a setting you missed. Pick a vision model in the same list for that turn.

What a month of this costs

Credits convert straight from the catalog price, and one credit is $0.00001.

On Qwen 3.8 Max, one million output tokens draws 495,100 credits. On A95B it is 600,000, because A95B is the dearer of the two on both input and output.

What you buy	Credits	Output tokens on Max	On A95B
€5 pack, no subscription	165,000	about 333,000	about 275,000
Hosted, €19 a month	900,000	about 1.8 million	about 1.5 million
Pro, €49 a month	2,350,000	about 4.7 million	about 3.9 million
Max, €99 a month	5,000,000	about 10.1 million	about 8.3 million

All of those are ceilings nobody reaches. Input is billed from the same pool at 0.165 credits per token on Max and 0.2 on A95B, a long conversation resends its whole history every turn, and images and video draw on the same balance. Treat the table as the shape of a budget, not a forecast.

The Think button is the difference between the two

On Max, reasoning is genuinely optional, and switching it off is the single largest lever on what a session costs. Leave it on for work that has to be derived. Turn it off for extraction, reformatting, translation and summarising, where you already know the shape of the answer and are only paying for the model to talk itself into it.

On A95B there is no such lever. It reasons every turn by design. There is no parameter that stops it, so plan for a reasoning pass in every single answer and choose the model deliberately rather than by habit.

One more thing about the effort control, since it looks like the same knob and is not: on these two, effort rungs are not what moves the reasoning. On Max it is the Think button. On A95B nothing moves it.

Picking between them

Max is cheaper per token on both sides and it lets you switch the reasoning off, which makes it the default of the two for most work. A95B is the open weight 2.4T checkpoint with 95 billion active per token and a 262,144 token context, and it always reasons, which is either the reason you want it or the reason you do not.

The honest summary: run the 27B at home when the material should not leave your machine, when you want it offline, and when you would rather pay once for a card than per token forever. Reach for a flagship when the problem is genuinely bigger than the hardware in the room. Most people who use both end up doing exactly that.

The full guide with the plan details is at how to run Qwen 3.8 without a GPU.

Uncensored AI chat online: text without a moderation layer, and the honest limits

David — Mon, 07 Sep 2026 17:43:50 +0000

Ours, said first: we make Locally Uncensored, a free open source desktop app for Windows and Linux that runs open models on your own machine, and LU Labs is the hosted layer next to it. So this is a guide written by an interested party, and the useful thing we can do about that is be precise instead of enthusiastic.

The phrase "uncensored AI chat" gets used to mean several different things, most of them oversold. This post says exactly what we do and do not do, because the gap between the marketing version and the real one is where people waste money.

What the word means here, for text

Hosted text models answer as they were trained. We do not put a moderation layer in front of them and we do not rewrite what comes back. That is the whole claim. It is smaller than the usual pitch and it is the part that actually changes your experience.

What it means in practice: no refusal injected by us on top of the model, no silent prompt rewriting, no filtered response substituted for the one the model produced. What the model does is the model's business, and models differ. A finetune tuned for open roleplay behaves differently from a general assistant, and a general assistant will still decline things on its own because that behaviour is baked into its weights. Nobody can sell you a model that never refuses. They can only stop adding refusals of their own, which is what we do.

The models people come for on the text side are Euryale 70B, Lunaris 8B and MythoMax 13B. They sit in the same catalog as Kimi K3, GLM 5.3, Qwen 3.8 Max and DeepSeek V4 Pro 0813, and that catalog is now the same everywhere: every plan carries all of it, and so does a credit pack bought with no subscription at all. Nothing is held back for a higher tier. What a plan buys is credits, not access.

What we do not claim, for images and video

This is the section most articles on this topic skip, so here it is plainly.

Hosted image and video generation follows our Terms of Service, and prompts are checked before anything is rendered. That check is real and it is not the same regime as text. If you are choosing a service on the assumption that hosted image work here is open the way the text side is, that assumption is wrong, and we would rather you know now than after a purchase.

We do not promise nudity or adult imagery, on any plan, at any price. There is no tier that changes this.

And one rule that has no exception anywhere in the product, hosted or local, text or image: content involving minors is refused. Always, everywhere, no argument, no context that makes it acceptable. If that is what you came for, leave.

The route that has no layer at all

If what you actually want is a model with nothing between you and it, run it yourself. That is the free desktop app, on Windows or Linux, with the weights on your disk and no account. No queue, no meter, no policy document, no network. It is free and stays free, and we built it first for exactly this reason.

The honest tradeoff is hardware. A 70B class model does not fit on a normal card, and the models people want most for this are large. So the split most users land on is local for anything private or offline, hosted for the sizes the machine refuses.

Using the hosted side

Browser: sign in at lu-labs.ai and open the Studio. Nothing to install, which is also the only route on a Mac, where we do not ship a desktop build.

From your own client: mint a key under Settings, then Cloud API keys. It starts with lu_ and it is shown once, because we store a hash rather than the key itself. The endpoint takes the OpenAI chat completions shape:

curl https://lu-labs.ai/api/inference/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"zai-org/GLM-5.3-Flash","messages":[{"role":"user","content":"Hello"}]}'

That makes SillyTavern the obvious client for this audience, and it works: custom OpenAI-compatible source, base URL https://lu-labs.ai/api/inference/v1, key in the key field, model id typed in verbatim. Aider, LibreChat and anything else with a base URL field work the same way.

One implementation note that matters if you use agents. Most of the roleplay finetunes do not ship a tool calling template, which normally means no function calling. We transport the tool definitions through the prompt instead and parse the calls back out, so agent workflows run on Euryale and Lunaris like they do on anything else. Thinking is per model rather than a global switch: Lunaris 8B has no thinking mode at all, Kimi K3 has a toggle, and GLM 5.3 Flash reasons on every turn with an effort setting instead of an on and off.

What it costs

Credits, one pool, one credit is $0.00001, and every model bills at its own rate rather than a blended average.

Hosted, €19 a month, 900,000 credits.
Pro, €49 a month, 2,350,000 credits.
Max, €99 a month, 5,000,000 credits.
Packs, no subscription needed at all. €5 for 165,000 credits, €10 for 350,000, €25 for 900,000. Credits never expire, and you can buy any pack on your first purchase.

To convert that into something real. Euryale 70B runs at 0.085 credits per output token, so a €5 pack is roughly 1.9 million output tokens on it. GLM 5.3 Flash is 0.05, which is about 3.3 million on the same pack. Kimi K3 is 1.425, which is about 115,000. For long chat sessions on the finetunes that is a lot of reading, and the same pack disappears in an afternoon if you spend it on the biggest reasoner in the list. Rates per model are on the pricing page.

The rest of the honest list

It is not private the way local is. A hosted prompt reaches a GPU that is not yours. Accounts and billing sit in the EU, we do not train on user data and we never sell it. That is a commitment, not a guarantee of physics. The guarantee of physics is the desktop app, offline.

The hardware is shared. Managed H100, A100 and B200 class GPUs, and at busy hours there is a short queue.

The catalog is curated. No bring-your-own checkpoint. If your setup depends on one specific community merge, run it locally.

It is metered. Credits, not unlimited. A pool tells you the truth about what a model costs; an unlimited label does not.

If the text side is what you were looking for, start with a Hosted month or a €5 pack. The longer version of this guide, with the policy wording in full, is at the uncensored AI chat guide.

An LM Studio alternative for Mac that needs nothing installed

David — Mon, 07 Sep 2026 17:43:29 +0000

This is a guide to our own product, so read it with that in mind. LU Labs is the hosted layer we run next to our open source desktop app for Windows and Linux, and on a Mac the hosted browser Studio is the only route we offer, because we do not ship a Mac desktop build.

LM Studio is good software. If it works on your Mac, use it. This post is for the Macs where it does not: the low memory machines, the work laptops with locked installs, and the situation where the model you want is simply larger than the memory you have. That last one is the common case and the one people are slowest to accept.

The memory wall, in plain terms

Apple Silicon has one memory pool shared between the CPU and the GPU, which is the reason a Mac punches above its price on local inference. It is also the reason a Mac hits a hard stop.

Take Qwen 3.8 27B, the dense open weight model most people try first. At 4-bit the weights alone are about 17 GB that has to sit in memory before the first token, plus the KV cache for your context, plus macOS, plus whatever your browser is holding. On a 32 GB Mac that is comfortable. On 24 GB it works if the machine is otherwise idle. Below that it does not load, and no download manager or quant picker changes that. LM Studio will show you the model, let you pull it, and then the machine will tell you no.

And that is the small end of what people ask for. Kimi K3 is 2.8 trillion parameters and Qwen 3.8 A95B is 2.4 trillion. Those do not run on any Mac that exists, at any quantization, and pretending otherwise wastes an afternoon and 40 GB of disk.

The honest options at that point are a bigger Mac, a much smaller model, or a machine that already has the card.

What the hosted route looks like on a Mac

Open lu-labs.ai in Safari or Chrome, sign in, use the Studio. That is the whole install process. No Homebrew, no Xcode command line tools, no Metal build, no .dmg you have to right-click open because it is not notarized, no multi gigabyte download sitting on a laptop SSD you were already fighting for space on.

Every plan reaches the same chat catalogue, and so does a credit pack bought without any subscription. There is no shortlist and no model held back for a higher price: Kimi K3, Qwen 3.8 Max, Qwen 3.8 A95B, GLM 5.3 and GLM 5.3 Flash, DeepSeek V4 Pro 0813 and DeepSeek V4 Flash 0731, DeepSeek V3.2, Gemma 4 26B and gpt-oss 120B all sit on the same list. What a plan buys is credits, not access.

Two things a local Mac setup usually cannot give you at all come with the same account: 10 image models and 5 video models. Flux 2 Dev, Flux Dev, Flux Schnell, Qwen Image, HiDream, HunyuanImage 2.1, Z-Image Turbo, Chroma, Prefect Pony XL and Neta Lumina on the image side, with inpainting on Flux Dev, Qwen Image Edit, background removal, an eraser and upscale to 2k, 4k or 8k. Video is Wan 2.2 720p, Wan 2.2 Fast, LTX-2 with audio, LTX 2.3 and HunyuanVideo 1.5, at 5 or 8 seconds a clip.

Keeping your existing tools

The part that matters if you already have a workflow: this is an OpenAI-compatible endpoint, so the Mac apps you point at LM Studio's local server can be pointed here instead by changing a base URL.

Mint a key in Settings, then Cloud API keys. It starts with lu_ and is shown once, since we store a hash and not the key.

curl https://lu-labs.ai/api/inference/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"moonshotai/Kimi-K3","messages":[{"role":"user","content":"Hello"}]}'

Or from Python:

from openai import OpenAI

client = OpenAI(
    base_url="https://lu-labs.ai/api/inference/v1",
    api_key="lu_...",
)

resp = client.chat.completions.create(
    model="zai-org/GLM-5.3-Flash",
    messages=[{"role": "user", "content": "Explain this stack trace."}],
)

Model ids are the exact catalog strings, moonshotai/Kimi-K3 and zai-org/GLM-5.3-Flash, not the friendly names from the picker. SillyTavern, Aider, LibreChat and anything else that takes a base URL work the same way. If your Mac tool has a field marked "OpenAI compatible", it is already compatible.

What it costs

Credits from one pool, one credit is $0.00001, and each model bills at its own rate rather than a blended average.

Hosted, €19 a month, 900,000 credits, plus LoRA training 2 a month.
Pro, €49 a month, 2,350,000 credits.
Max, €99 a month, 5,000,000 credits.
Packs, no subscription. €5 for 165,000 credits, €10 for 350,000, €25 for 900,000. Credits never expire and you can start on any pack.

Anchors, since credits are abstract until you convert them. GLM 5.3 Flash bills 0.05 credits per output token, so a €5 pack is roughly 3.3 million output tokens there. Kimi K3 bills 1.425 credits per output token, so the same pack is about 115,000 output tokens. That spread of nearly 30x is the reason the catalogue is open everywhere: the meter decides what a model costs you, not the plan you are on.

On the image side, Flux Schnell is 300 credits an image and Flux 2 Dev is 1,200, which makes the €5 pack about 550 quick drafts or 137 Flux 2 Dev renders, and the Hosted month about 3,000 or 750. The pricing page carries the full rate card.

The honest comparison

	LM Studio on your Mac	Hosted in the browser
Install	Yes, plus the model download	None
Cost	Free	€19 a month, or a €5 pack
Works offline	Yes	No
Prompt stays on the machine	Yes	No
Model size ceiling	Your unified memory	The catalog
Images and video	Separate tooling	Included
Custom checkpoints	Anything you can download	Curated catalog only

Read that table honestly and it tells you which one you want. If your Mac has the memory for the models you actually use, LM Studio is free and private and there is no argument for paying us. If it does not, or if you also want image and video work, or if you cannot install software on the machine at all, the browser route is the one that exists.

The limits on our side, stated rather than buried: the hardware is shared, so at busy hours there is a short queue. It is managed H100, A100 and B200 class GPUs, which is more card than any Mac. Accounts and billing are in the EU, we do not train on user data and we never sell it, but a hosted prompt still reaches a GPU that is not yours, and no privacy policy makes that untrue. The catalog is curated, so there is no bring-your-own checkpoint. And there is no Mac desktop build, which is why this article exists in the first place rather than a download link.

Start with a Hosted month or a €5 pack, or read the longer version at how to use LU Labs Cloud on a Mac.

Flux 2 Dev without a 24 GB card: rendering online from €5

David — Mon, 07 Sep 2026 17:43:08 +0000

Disclosure first, because this is a guide to our own thing: LU Labs is the hosted layer we built next to our open source desktop app for Windows and Linux, which is free and runs models locally. This post is about the hosted image side, what a render costs in credits, and where the hosted route is the wrong choice.

The hardware problem, briefly

Flux 2 Dev is a large model. Running it locally at a quality you would actually ship means a 24 GB card, and the workarounds below that line are the usual ones: heavy quantization, sequential CPU offload, or a tiling scheme that turns a render you would have watched into a render you go and make coffee during. All of them work. None of them are fun on a laptop.

The alternative is to keep the prompt and send the compute elsewhere. That is what this is.

The route

Sign in at lu-labs.ai and open the Studio in your browser. Nothing to install, no Python environment, no xformers version to reconcile. Every hosted account gets the same 10 image models:

Flux 2 Dev, the one in the title
Flux Dev and Flux Schnell, the older pair, Schnell being the fast draft model
Qwen Image
HiDream
HunyuanImage 2.1
Z-Image Turbo
Chroma
Prefect Pony XL
Neta Lumina

The image catalog does not scale with the plan. It is identical on every hosted plan and identical on a pack, and the chat catalog works the same way, so the only thing price changes is how many credits you get.

Working, not just generating

A first render is rarely the last one, so the editing tools matter more than the model count:

Inpainting on Flux Dev. Mask the part that is wrong, describe what should be there instead.
Qwen Image Edit. Instruction-driven edits on an existing image rather than a fresh sample.
Background removal and an eraser for the cleanup passes that otherwise send you back to a desktop editor.
Upscale to 2k, 4k or 8k, which is the step people forget until they try to print something.

The workflow that gets the most out of a balance is boring and effective: draft wide on Flux Schnell until the composition is right, then re-render the keeper on Flux 2 Dev, then inpaint the one detail that is off, then upscale. Cheap exploration, expensive commitment, in that order.

What it costs, in whole numbers

Credits, one pool, one credit is $0.00001. Images are priced per render at a fixed rate per model:

Model	Credits per image
Flux Schnell	300
Flux Dev	1,200
Flux 2 Dev	1,200

That converts cleanly:

	Flux Schnell renders	Flux 2 Dev renders
€5 pack, 165,000 credits	about 550	137
Hosted month, 900,000 credits	about 3,000	750

The €5 pack is the entry point in the title and it is a real one. No subscription is required for it, the credits never expire, and you can buy any pack on the first purchase: €5 for 165,000 credits, €10 for 350,000, €25 for 900,000. If you render in bursts, a pack is the honest shape, because there is no month to waste.

The Hosted plan is €19 a month for 900,000 credits, and it adds two things a pack does not have: LoRA training, 2 a month, and a monthly refill you do not have to think about.

The full rate card is on the pricing page.

The same balance does video

Worth knowing before you decide how much to buy, because video is where a balance disappears fastest. Every hosted account also gets 5 video models: Wan 2.2 720p, Wan 2.2 Fast, LTX-2 with audio, LTX 2.3 and HunyuanVideo 1.5, with clips at 5 or 8 seconds. An LTX-2 clip at 5 seconds is 8,000 credits, a different order of cost from a still image. Plan for that, or keep video and stills on separate purchases.

Where the hosted route is wrong

You want a specific community checkpoint. The catalog is curated and there is no bring-your-own checkpoint. If your work depends on one particular merge from a model site, hosted will not do it and local will.

You want the render to never leave your machine. Hosted means the prompt reaches a GPU that is not yours. Accounts and billing sit in the EU, we do not train on user data and we never sell it, but that is a policy rather than physics. The free desktop app on Windows or Linux is the version where the guarantee is physical, and that is exactly why we shipped it first.

You render constantly. If a 24 GB card is already in the machine and it is busy every day, local is cheaper than any credit pool, and it always will be. Hosted wins on the days your hardware says no, on a laptop, or on a Mac where we have no desktop build and the browser Studio is the only route.

One more honest limit: the hardware is shared. It is managed H100, A100 and B200 class GPUs, which is more card than most people have, but at busy hours there is a short queue. A render that normally comes back quickly sometimes waits.

The short version

If you have the card, run Flux 2 Dev locally. If you do not, a €5 pack gets you 137 Flux 2 Dev renders or about 550 quick drafts, with no subscription and no expiry date on the credits, and the editing and upscale tools come with it rather than as a separate purchase.

Start with a Hosted month or a €5 pack and the longer guide is at how to generate Flux 2 images online.

An OpenAI-compatible endpoint with 38 open models, including the unrestricted ones

David — Sun, 06 Sep 2026 09:39:18 +0000

We built LU Labs as a desktop app that runs open models on your own machine. That part is free and stays free. But a good share of the people who downloaded it wrote back with the same sentence: my laptop cannot hold a 400B model, can you just run it.

So we run it. Same catalog, same tools, someone else's GPUs. This post is the developer version of the pricing page: what the endpoint is, what the numbers mean, and where it stops.

The endpoint

It speaks the OpenAI chat completions shape. Point any client at it:

from openai import OpenAI

client = OpenAI(
    base_url="https://lu-labs.ai/api/inference/v1",
    api_key="lu_...",
)

resp = client.chat.completions.create(
    model="zai-org/GLM-5.3-Flash",
    messages=[{"role": "user", "content": "Explain the DSA attention change in GLM-5.3."}],
)

Keys are minted in account settings, up to five of them, format lu_ plus 40 hex characters. We store a SHA-256 hash and nothing else, so the plaintext exists in the create response and never again. Worth saying plainly because it is the part people get wrong when they build this themselves: the key authenticates for inference only. It cannot read your account and it cannot mint more keys. A leaked key can spend your plan's credits, which is bad, and that is the whole blast radius, which is the point.

/models returns what your plan can actually reach, not a catalog you then get 403s from.

The catalog

53 models: 38 for chat and code, 10 for image, 5 for video.

Every plan gets the same 15 chat models. That shortlist is not the cheap leftovers. It includes Kimi K3, Hermes 3 405B, Qwen3 Coder 480B and GLM 5.3 Flash, plus all five of the unrestricted ones. Pro and Max add the remaining 23 for the full 38, among them GLM 5.3, Qwen 3.8 A95B and DeepSeek V4 Flash 0731.

Two properties that are easy to promise and annoying to deliver, so here is where we stand on both:

Tools work on all of them. Including the unrestricted models, which mostly do not ship with a tool-calling template. For those we transport the tool definitions through the prompt instead of the API field, which means agent mode and the coding agent run on Hermes and Euryale the same as on anything else.

Thinking is per model, not a global switch. Some models always think, some never do, some take a toggle, and GLM 5.3 Flash takes an effort level. The /models response carries that per entry, so a client can render the right control instead of guessing.

Image, video and every Create tool are identical on every hosted plan. The chat catalog is the only thing that scales with price.

What a euro buys

Credits, not a flat rate. One monthly pool, spend it on whatever you want.

	Hosted	Hosted Pro	Hosted Max
Per month	€19	€49	€99
Credits	900,000	2,350,000	5,000,000
Chat models	15	38	38
Images, if you spend it all there	3,000	7,800	16,600
Video clips, same	60	156	333
Output tokens, cheapest to priciest model	0.6M to 225M	1.6M to 587M	3.5M to 1250M
LoRA trainings	2	8	20

Yearly billing is two months free. Top-ups start at €5 for 165,000 credits, they never expire, and they are spent only after the month's plan credits are gone, so a top-up is never wasted by a quiet month.

The token range is wide because it is a real range. 225M output tokens is Llama 3.1 8B Turbo, which works out to about €0.08 per million output tokens on the €19 plan. 0.6M is Kimi K3, about €30 per million. Same pool, and you choose. The credits follow each model's own rate rather than a blended average, because a blended average is how a bill surprises you.

The four things it does not do

It is not GPT or Claude. Open weights only, by choice. If your evaluation depends on a closed frontier model, this is the wrong endpoint.

It is not unmetered. Cache hits bill at the cached rate rather than list, and a run you abort stops billing at the abort. Both of those were bugs first and are now tests. But there is still a meter.

It is not private the way local is. Local mode on your own machine is the private one, and it is free forever for exactly that reason. Hosted means your prompt reaches a GPU that is not yours. We never train on it and never sell it, account data sits in the EU and you can delete it, and that is a promise rather than a physical guarantee. If you need the physical guarantee, download the app and run it offline.

It is not a lock-in. The catalog is open weights, so every model you use here you can also pull and run yourself. Export your chats and go. That is not generosity, it is the same argument that made us build the local app first.

Why we did it this way

The honest reason for credits instead of tiers-with-limits: we priced the pool off real wholesale cost, and a model that costs 350 times more per token than another cannot sit behind the same "unlimited" label without one of us lying. A pool tells you the truth and lets you spend it where you want, including all of it on video if that is what you came for.

Local stays free. The cloud is for the days your hardware says no.

Pricing and the full model list · Download the local app

Run GLM-5.3 Locally: Real Quant Sizes, the llama.cpp Surprise, and the reasoning_effort Trap

David — Sat, 05 Sep 2026 16:25:14 +0000

Z.ai published the GLM-5.3-Flash weights on 27 August 2026 at 10:33 UTC and the GLM-5.3 flagship on 28 August 2026 at 15:22 UTC. The models were on Z.ai's own API first, though I could not find a primary source that dates that launch, so I am not going to put a day on it. I spent the evening reading the actual repositories instead of the announcement. Here is what is worth knowing before you start a download measured in hundreds of gigabytes.

TL;DR

Two models: GLM-5.3 at 753.9B params, weights out 28 August, and GLM-5.3-Flash at 320.8B (18B active), weights out 27 August.
Smallest usable builds: 216.7 GB flagship, 93.1 GB Flash.
The flagship runs in stock llama.cpp. Flash does not yet. Yes, that way round.
reasoning_effort defaults to max, including when you typo it.
Flash is MIT. The flagship is not.

The architecture surprise

I expected the smaller model to have better support. The opposite is true, and the reason is interesting.

I diffed the two config files:

Field	GLM-5.2	GLM-5.3
`architectures`	`GlmMoeDsaForCausalLM`	`GlmMoeDsaForCausalLM`
`num_hidden_layers`	78	78
`hidden_size`	6144	6144
`n_routed_experts`	256	256
`num_experts_per_tok`	8	8
`max_position_embeddings`	1048576	1048576
Params (GGUF metadata)	753,864,139,008	753,864,139,008

Identical. Every field, and the parameter count down to the last digit. GLM-5.3 is GLM-5.2 with different post-training, exactly as Z.ai described it at the API launch.

So the flagship's GGUF architecture is glm-dsa, which llama.cpp has supported since GLM-5.2:

$ grep 'LLM_ARCH_GLM' src/llama-arch.cpp
{ LLM_ARCH_GLM4,     "glm4"     },
{ LLM_ARCH_GLM4_MOE, "glm4moe"  },
{ LLM_ARCH_GLM_DSA,  "glm-dsa"  },

Flash is the genuinely new architecture. It reports itself as glm5next, which is still not in the main branch on 2 September 2026. PR 27754 is open, and Unsloth's card points at it or at their desktop app.

If you get an unknown architecture error loading a Flash GGUF, that is this. Check whether the PR has landed before you spend an hour rebuilding, because this will change.

Actual sizes

Unsloth packs, read on the evening of 28 August 2026. The Flash pack is dated 26 August 2026 and the flagship pack 28 August, which is worth noting because a GGUF cannot predate the weights it was built from, so 26 August is the lower bound for Flash. MLX builds of the flagship went up on 2 September 2026 as mlx-community/GLM-5.3-4bit.

GLM-5.3

UD-IQ1_S     216.7 GB
UD-IQ1_M     228.5 GB
UD-Q2_K_XL   253.9 GB
UD-Q3_K_XL   343.0 GB
UD-Q4_K_XL   467.3 GB

GLM-5.3-Flash

UD-IQ1_S      93.1 GB
UD-IQ1_M      97.6 GB
UD-Q2_K_XL   108.7 GB
UD-IQ3_XXS   120.4 GB
UD-Q3_K_XL   147.5 GB
UD-IQ4_XS    156.8 GB
UD-Q4_K_XL   199.7 GB

These are file sizes. Your KV cache is extra, and on a model advertising 1,048,576 tokens of context that is not a rounding error.

Flash is natively multimodal, and the vision half is a separate 1.13 GB file:

mmproj-F16.gguf    1.13 GB
mmproj-BF16.gguf   1.16 GB

Skip it and Flash silently cannot see images. There is no warning.

Running it

Flash, per Unsloth's card, once the PR is in your build:

llama serve -hf unsloth/GLM-5.3-Flash-GGUF:UD-Q4_K_XL

The flagship works on a stock build today, if you have somewhere to put 216.7 GB.

The default that will bite you

GLM-5.3 takes reasoning_effort with three values: low, high, max.

From the model card: it defaults to max if not passed, or if set to any other value.

There is no middle default, and an invalid value does not error, it silently gives you the most expensive setting. So a typo in that field buys you a long think on every message. Start at low.

One caveat, because I measured it rather than read it. Through a hosted OpenAI-compatible provider the behaviour is not the same: on the same arithmetic prompt the flagship spent 4 output tokens at low, 10 at medium, 11 at high and 48 at max, and sending no field at all landed at 30, not at max. medium is honoured there as a real middle rung even though the card does not list it. So the model card describes the model, and whatever sits in front of it may have its own opinion. Measure your own stack before you trust either.

Licensing, which went backwards

GLM-5.2: MIT
GLM-5.3-Flash: MIT
GLM-5.3 flagship: Z.ai's own glm-5.3 licence, not MIT

Same architecture and same parameter count as the MIT-licensed 5.2, different terms. If you are shipping something commercial, that is a real decision and not a formality.

Z.ai was open about the reasoning: they held the weights back for safety evaluation, and named the concern. Their own numbers put ExploitBench at 54.4 percent versus 24.4 for GLM-5.2, and CyberGym at 84.5 versus 77.2. Those are vendor figures from a vendor harness, unreproduced by anyone outside the company at the time of writing, and the delay makes more sense once you see which benchmark moved most.

If the hardware is the blocker

Both of these are now in the LU Labs Cloud catalog, which is the product I work on, so read this section with that in mind. GLM-5.3-Flash sits on the Hosted plan, so it is on every plan, and the GLM-5.3 flagship is on Pro and Max. The catalog is served to the clients rather than compiled into them, so they appear in the model picker in the web app and in the desktop app without anyone installing an update.

Both models think before every answer, so there is now an Effort button next to the Brain button, and it controls how much the model thinks. It is in the web app today; on the desktop it arrives with the next update. Most reasoning models in the catalog offer three settings, Low, Medium and High, and these two add a fourth above them, Max. It starts on High. A higher setting means more output tokens and therefore more credits, so Low is the saving. Tools run natively on both, and Flash takes images.

What I did not check

The benchmarks. I have not run them and neither has anyone else independently yet. Everything above about sizes, parameters, architectures and licences comes from the HuggingFace API, the config files and the llama.cpp source, all of which answer in seconds and none of which are a press release.

If you are picking a model to actually run this week and you do not have 128 GB of memory, none of this is your model. That is a legitimate answer too.

We Benchmarked Our Agent Against opencode: Same Task, Same Model, 40 Percent Fewer Credits

David — Sun, 23 Aug 2026 00:11:55 +0000

Every coding agent says it is efficient. Almost none of them publish the bill. So we ran the boring experiment: the same bugfix, the same model, the same API, the same prices, and a byte identical prompt, once through opencode and once through the coding agent inside Locally Uncensored.

Headline: opencode averaged 2157 credits over three runs. Our 2.6.6 agent finished the identical task for 1298. That is about 40 percent less, and even the cheapest opencode run came in 29 percent above our number.

The interesting part is not the headline. It is why the gap exists, and it is not the reason most people guess.

Setup

A cost comparison is only worth reading if everything that drives cost is nailed down. What was held constant:

Held constant	Value
Task	Fix a failing test in a small npm repo, then commit
Repository	Three files, a one line bug in `add.js`, tests red at the start
Prompt	Byte identical, sha256 `29cec6c3...cf62687`
Model	`deepseek-ai/DeepSeek-V3.2`
Endpoint	The same OpenAI compatible API for both agents
Prices	Same account, same tier, same per token rate
Counting	One wire proxy in front of the API, credits read before and after every run
opencode	1.18.21 from npm, wired as an OpenAI compatible provider, `opencode run --auto`, otherwise defaults

Success was defined before the runs, not after:

npm test passes
exactly one commit, with the required message
only add.js changed
clean working tree at the end

All four runs cleared that bar. Nothing failed, so cost is the only variable that moved.

The numbers

Run	Credits	Requests	Prompt tokens	Success
opencode, run 1	1679	8	98,789	yes
opencode, run 2	2433	11	146,058	yes
opencode, run 3	2358	11	146,387	yes
Locally Uncensored 2.6.6	1298	16	74,629	yes
Locally Uncensored 2.6.5	4395	30	257,270	yes

Read the last row first. Our own shipped agent from one release earlier is the most expensive thing in that table, by a lot. This is not a chart built so that we win by construction. It is a chart that shows what one efficiency pass is worth, and the previous version of our own software is the loser in it.

Why the gap exists

The tempting explanation is that one agent is smarter and needs fewer steps. That is not what happened, and the direction is the reverse of what you would expect.

opencode used fewer turns than we did. Eight to eleven requests against our sixteen. If you scored this on steps, opencode wins. The bill went the other way because of what every single request carries.

Per request	opencode	Locally Uncensored 2.6.6
Prompt tokens per request	12,349 to 13,308	4,664
Tool catalogue size	21,188 bytes	7,703 bytes
Credits per prompt token	0.01700 / 0.01666 / 0.01611	0.01739

That last row is the honest one. The billing rate is the same. Credits per prompt token land within a few percent across all four runs, and ours is marginally the highest of the set. Nobody got a secret discount. The entire difference in the invoice is token volume, not token price.

Two things drive that volume, and both are familiar to anyone who has built an agent loop:

The fixed block. A tool catalogue of 21,188 bytes against 7,703 bytes is roughly three times the standing overhead, and you pay it again on every call in the loop, whether the model touches those tools or not.
Context decay. As the agent works, the transcript grows. Old tool output that stopped mattering ten steps ago keeps getting resent at full length unless something actively trims it.

Put together, a fixed block that size pushes every opencode request past 12,000 tokens. Six agent steps at that weight already approach our total consumption for the entire task.

What changed between our own 2.6.5 and 2.6.6

The 4395 in the table is not a strawman we built for the article, it is what we shipped in the previous release. Between 2.6.5 and 2.6.6 we went after exactly the two items above: how big the fixed block is, and how much of the transcript gets resent. Measured over our own set of tool driven runs, Locally Uncensored 2.6.6 spent 78.6 percent fewer credits than Locally Uncensored 2.6.5, and on the longest run in that set 80.4 percent fewer than 2.6.5. Both figures compare our own agent against our own previous release. Neither of them is the gap to opencode, which is the roughly 40 percent above. The two also come from different sets: on the single task in the table above, our own 2.6.5 would have cost 4395 credits against 1298, which is 70 percent. The opencode comparison is simply what fell out when we pointed the same measurement at somebody else's loop.

Limits of this benchmark

This is where vendor benchmarks usually go quiet, so here is how far the number actually carries.

One scenario. A tiny repo and a one line bug. It says nothing about a large codebase, a multi file refactor, or a session that runs for an hour. We did not run those.
Uneven sample. opencode ran three times, we ran once. The spread inside opencode alone is 45 percent, from 1679 to 2433. Our own spread is unknown. One run is a data point, not a distribution.
Default settings. opencode ran as it ships. It is configurable, and a tuned setup with a trimmed tool set would land somewhere else. We did not tune it in either direction.
opencode is free software. The tool costs nothing. Everything measured here is the model bill, which you pay to whichever provider you point at. This is token efficiency, not licence fees.
Cost is not quality. Every run in the table produced correct, committed work. On a harder problem the ranking could look different, and cheapest is never automatically best.

What the measurement does support is a narrower claim than the headline: on short, well scoped agent tasks, opencode can hardly land below us, because the fixed per request overhead sets a floor. Even its best run, with only eight requests, still needed 98,789 tokens.

Credit where it is due

opencode finished the job three times out of three, took fewer turns than we did, and produced a clean diff with the right commit message every time. It is a genuinely good agent and it is open source. Nothing here is an argument to stop using it.

It is an argument to measure your own loop. Agent bills are made of tokens you never see, and two tools that both feel fast can be a factor of 1.66 apart on the invoice. If you build agents, the two numbers worth putting on a dashboard are prompt tokens per request and the byte size of your tool catalogue. They predict the bill better than step count does.

Full writeup with the methodology and the raw counts: opencode Alternative: We Measured the Cost per Task. The agent lives inside Locally Uncensored (AGPL, free), and the hosted models we benchmarked against sit in LU Labs Cloud if you want them in one picker. You can also point it at a model on your own GPU and skip the API bill entirely.

FAQ

Is opencode expensive? opencode is free. The bill is the model. On this one line bugfix with DeepSeek V3.2 the three runs cost 1679, 2433 and 2358 credits, a 45 percent spread between cheapest and dearest.

Why does it use so many tokens? Not through extra steps, it used fewer than we did. Each request carries more: 12,349 to 13,308 prompt tokens against our 4,664, with a tool catalogue of 21,188 bytes against 7,703 resent on every call.

Is the billing rate really identical? Yes, and that is the point. Credits per prompt token came out at 0.01739 for us and 0.01700, 0.01666, 0.01611 for opencode. The gap is volume, not price.

Can I reproduce it? Three file npm repo with a red test, prompt pinned at sha256 29cec6c3...cf62687, model deepseek-ai/DeepSeek-V3.2, opencode 1.18.21 at defaults, requests counted through a wire proxy, credits read before and after each run.

Run Qwen 3.8 27B Locally: Real GGUF Sizes, the KV Cache Trick, and the Template Trap

David — Fri, 14 Aug 2026 20:48:03 +0000

Qwen 3.8 arrived as two different releases with two different licences, and only one of them is something you can put on a card you own. The 2.4 trillion parameter A95B opened up on 12 August under Alibaba's own qwen3.8-max terms. The one that matters for local work is Qwen 3.8 27B, whose safetensors went up on 13 August at 08:23 UTC with an Apache 2.0 LICENSE file following the next morning. Both dates are off the Hugging Face commit log, not a launch post.

Here is the practical picture: what it needs, why its long context is unusually cheap, and the one setting that makes people think they downloaded a broken quant.

The shape of the model decides everything

27B dense parameters across 64 layers, hidden size 5120. The interesting part is in config.json, where layer_types reads 48 linear attention layers and 16 full attention layers, alternating three to one (full_attention_interval: 4). Only those 16 layers keep a KV cache.

The rest of the shape: 24 attention heads with head_dim 256 and 4 KV heads, a 248,320 token vocabulary, and max_position_embeddings of 262,144. It is a native vision language model, so images and video go in without a wrapper, and the ggml-org pack also ships a multi token prediction head as a separate file.

The numbers

Sizes below are the file sizes Hugging Face reports for unsloth/Qwen3.8-27B-GGUF, read on 14 August 2026. Packs differ by a few hundred megabytes, so check the repo you actually pull from. lmstudio-community has Q4_K_M at 16.8 GB and ggml-org at 19.0 GB for the same nominal quant.

Quant	Size on disk	Realistic home
UD-IQ2_XXS	9.0 GB	12 GB cards, visible quality cost
UD-Q2_K_XL	10.7 GB	12 GB cards, almost no context left
UD-Q3_K_XL	13.4 GB	16 GB cards
Q3_K_M	13.8 GB	16 GB cards
IQ4_XS	15.7 GB	largest quant that stays whole on 16 GB
Q4_K_M (sweet spot)	17.1 GB	24 GB cards
Q5_K_M	19.8 GB	24 GB, less context headroom
Q6_K	22.9 GB	24 GB barely, or 32 GB
Q8_0	29.0 GB	32 GB or a two card split
BF16 (from `ggml-org`)	53.8 GB	server cards, or CPU and patience
mmproj	0.9 GB	the vision encoder, separate file

Machine classes, honestly:

24 GB GPU: Q4_K_M whole, with real context headroom. This is the card the model was sized for.
16 GB GPU: IQ4_XS fits whole; Q4_K_M works with a few layers offloaded and costs you speed.
12 GB GPU: only the 2-bit quants, and you will feel it. A 3060 runs it, slowly.
Apple Silicon: 32 GB unified memory is comfortable at Q4_K_M, 24 GB works if nothing else is open.

Where the hybrid layout pays off

Every full attention layer stores a KV cache that grows with the context. At fp16 one token costs 2 x 4 heads x 256 dim x 2 bytes = 4 KB per layer. A conventional 64 layer model pays that on all 64 layers, which is 256 KB per token. Qwen 3.8 27B pays it on 16 layers, so 64 KB per token.

Context	KV cache	Q4_K_M total
8K	0.5 GB	17.6 GB
32K	2.0 GB	19.1 GB
128K	8.0 GB	25.1 GB
262K	16.4 GB	33.5 GB

That is the difference between a long window on the spec sheet and one you actually turn on. A 128K session that would need 32 GB of cache on a dense model needs 8 GB here.

With a current llama.cpp:

llama-server -m Qwen3.8-27B-Q4_K_M.gguf \
  --jinja \
  -ngl 99 \
  -c 32768

The template trap

That --jinja flag is not optional, and it is the single biggest source of "this quant is broken" reports.

Qwen 3.8 ships its own chat template. Load the model without it and the model has no reliable marker for where your turn ends and its answer begins. The two failure modes both look like a bad conversion: either it rambles past the stop token, or it answers in a clipped voice and loses the conversation between turns.

There is a second, sharper version of this. The official template wraps each assistant turn in a think block even when the reasoning is empty, then opens another one when generation starts. Across several turns those nest and the history gets truncated. Several GGUF packs already ship a corrected chat_template.jinja. If you converted the weights yourself, swap the template before you blame the quantization.

Vision needs the second file

The vision encoder is not inside the language GGUF. Download the mmproj file, about 0.9 GB, and load it alongside:

llama-server -m Qwen3.8-27B-Q4_K_M.gguf \
  --mmproj mmproj-F16.gguf \
  --jinja -ngl 99

Skip it and you have a strong text model that will politely tell you it cannot see the image you just pasted.

Thinking is a dial, not a switch

The model reasons before answering by default. Lower reasoning_effort from the default to medium or low for quick answers, or pass chat_template_kwargs with enable_thinking: false to skip the reasoning pass entirely. Locally the cost of thinking is not money, it is your own time watching tokens arrive, which is the more annoying currency.

The no-terminal route

If you would rather click than type flags, Locally Uncensored (open source, AGPL) wraps this: install, open the Model Manager, paste a Qwen 3.8 27B GGUF repo, pick the quant that fits your card, chat. It carries the llama.cpp engine, handles the offload split and the template, and keeps everything on your machine with no account and no telemetry.

And the 2.4T reality check

The big Qwen 3.8, Qwen/Qwen3.8-2.4T-A95B, has open weights but not an open licence. Hugging Face reports license: other with license_name: qwen3.8-max, so read the terms before you build a product on it, and do not repeat the "Apache" line that has been going around. At 2.4 trillion total parameters with 95 billion active it is a data center model regardless.

The split that works: run the 27B at home for anything private, offline, or repetitive, and reach the A95B through a hosted API when a task genuinely needs that scale. It is on DeepInfra, and both it and the 27B class of models sit in LU Labs Cloud if you want them next to each other in one picker.

FAQ

Is 27B enough? For local work it is the interesting size: big enough for real coding and agent loops, small enough to sit on one consumer card. The hybrid attention means long context does not price you out the way it does on a dense model of the same parameter count.

Ollama or LM Studio? Both pick up the shipped template automatically, which spares you the trap above. The bare-GGUF-plus-llama.cpp route is where people get bitten.

Uncensored builds? Apache 2.0 makes finetunes and abliterations fully legal, and community variants started appearing within a day of the weights. The stock model carries standard alignment.

How do I check the sizes myself? https://huggingface.co/api/models/<repo>/tree/main returns every file with its byte count. That is where every number in this post came from.

Run Ling 3.0 Flash Locally: 124B of Knowledge on a 96 GB Machine

David — Sun, 09 Aug 2026 12:56:06 +0000

The two big open releases on everyone's feed right now are Kimi K3 (2.8 trillion parameters, the first open 3T-class model) and Ling 3.0 Flash from Ant Group's inclusionAI. Only one of them can live on hardware a person owns, and it is not the one with the bigger headline. Here is the practical picture for running Ling 3.0 Flash locally, with measured file sizes instead of projections, plus the honest math on why K3 stays in the cloud.

The shape of the model decides everything

Ling 3.0 Flash is a 124B parameter Mixture of Experts that activates only 5.1B parameters per token (8 of 512 routed experts plus one shared). As with every MoE, total parameters set your memory bill and active parameters set your speed. 124B of knowledge at the memory counter, 5.1B of compute at the speed counter: that combination is why this model runs interactively on machines that would crawl under a dense 70B.

Two more architecture notes that matter in practice. It uses hybrid linear attention (35 Kimi Delta Attention layers alternating with 7 gated MLA layers), so long contexts grow memory gently instead of quadratically. And it is a native hybrid reasoner: it thinks before answering by default, and the thinking pass can be switched off per request, so you decide when to pay for reasoning.

The license is plain MIT. Weights are on Hugging Face under inclusionAI/Ling-3.0-flash, including official fp8, fp4 and int4 variants.

The numbers

Community GGUF conversions are up, and these sizes are measured from the repos, not estimated:

Quant	Size on disk	Realistic minimum
IQ1_S / IQ1_M	27 to 30 GB	36 GB RAM, visible quality cost
IQ2_M / Q2_K_XL	42 to 43 GB	48 to 64 GB
IQ3_XXS	51 GB	64 GB
Q4_K_M (sweet spot)	78 GB	96 GB
Q5_K_M	92 GB	128 GB
Q6_K	105 GB	128 GB
Q8_0	136 GB	192 GB

Realistic machine classes:

A Mac with 96 or 128 GB unified memory runs Q4_K_M comfortably, and 64 GB machines run the 2-bit and small 3-bit quants whole.
A 24 GB GPU plus 64 to 96 GB of DDR5 works well with MoE offload: attention layers and the shared expert on the GPU, routed experts in system RAM. Because only 5.1B parameters fire per token, generation speed stays in usable double digits.
32 GB or less: skip it and run a model that actually fits. The IQ1 files exist, but you will not enjoy them.

With a current llama.cpp:

llama-server -m Ling-3.0-flash-Q4_K_M.gguf \
  --ctx-size 32768 \
  --n-gpu-layers 99 \
  --n-cpu-moe 40

Start with a modest context window; the model was trained out to 256K, but every token of window is memory you could spend on a better quant instead.

The no-terminal route

If you would rather click than type flags, Locally Uncensored (open source, AGPL) wraps the whole thing: install, open the Model Manager, paste a Ling 3.0 Flash GGUF repo, pick the quant that fits your memory, chat. It handles the llama.cpp engine, the offload split, and keeps everything on your machine with no account and no telemetry.

And the Kimi K3 reality check

K3 deserves its headlines: first open 3T-class release, 1M token context, native image input, and it shares the Kimi Delta Attention lineage with Ling. But the smallest usable GGUF conversion, a 1-bit quant, is 466 GB on disk. The 2-bit is 861 GB. Q4 is 1.5 TB. Expert-pruned community builds squeeze toward the 512 GB server class, which is heroic and still not a gaming PC.

So the honest split for the price of one search query: run Ling 3.0 Flash at home, and reach K3 through a hosted API when a task genuinely needs the 1M window or vision. DeepInfra serves both (Ling at $0.03 in / $0.07 out per million tokens, which rounds to free; K3 at $2.85 / $14.25), OpenRouter lists a free Ling tier, and both landed in LU Labs Cloud on every plan this week, with reasoning as a toggle in both cases.

FAQ

Is it actually good? inclusionAI benchmarks it at parity with their previous 1T-class flagship on SWE-Bench Pro, agentic tool suites and long-context tasks. Vendor numbers, as always, but the architecture math is real, and the price of testing that claim yourself is a weekend download.

How fast is it locally? Speed tracks the 5.1B active count, not the 124B total. On Apple Silicon at Q4 and on GPU-plus-RAM hybrids, expect double-digit tokens per second.

Uncensored builds? None yet. The MIT license makes finetunes and abliterations fully legal, and a 5.1B-active model is cheap to tune, so expect community variants. The stock model carries standard alignment.

How to Run Qwen 3.8 Locally: The 2.4T Max Math, the Open Weight 27B, and What Runs Today

David — Mon, 03 Aug 2026 15:43:05 +0000

Alibaba made Qwen 3.8 Max generally available today (August 3): roughly 2.4 trillion parameters as a Mixture of Experts, a 1 million token context window, image and video input, and launch scores like 92.6 on GPQA Diamond and 86.6 on Terminal-Bench 2.1. The part that matters for this post: the open weight checkpoint is promised for the coming week, and it brings a consumer sized Qwen 3.8 27B along with it. Here is the practical picture for anyone who wants Qwen 3.8 on their own hardware.

The shape of the model decides everything

MoE models split the hardware question in two. Total parameters (2.4T) set your memory bill: every expert has to live somewhere. Active parameters (~95B per launch coverage) set your speed. That is why the Max serves cheaply in a datacenter, and also why it will never fit in your tower: all 2.4T parameters must sit in memory, because you never know which experts the next token activates.

The numbers for the Max

FP8, the native serving precision: ~2.4 TB of weights.
Q4, the local standard: ~1.2 TB.
An extreme 2 bit quant: ~600 GB, with real quality loss.

The largest single machine a consumer can buy in 2026 holds 512 GB of unified memory, less than half of the Q4 weights, before you allocate a single byte of KV cache for that million token context. Someone will chain Mac Studios together for a single digit tokens per second demo within weeks of the weights dropping. It will be a great video and a bad daily driver.

The 27B is the actual local story

This is the difference between this launch and Kimi K3 or DeepSeek V4 Pro: the same drop includes an open weight 27B. Projected from the Qwen 3.6 27B precedent:

VRAM	Quant class	Precedent size
8 GB	2 bit (UD-IQ2 class)	8.7 GB, reduced quality
12 GB	Q3_K_M	13 GB, RTX 3060 sweet spot
16 GB	Q4_K_M, recommended	~16 GB
24 GB	Q6_K, near lossless	~21 GB

Community GGUFs of open Qwen releases usually appear within days of the weights. If the 27B inherits even part of the Max's agentic gains (FrontierSWE jumped from 40.7 to 73.5 this generation), it becomes the default local model in its class more or less immediately. And weights mean derivatives: every open Qwen generation has received abliterated and heretic builds within weeks.

What to run today, not next week

Until the drop, the newest Qwen you can actually run is Qwen 3.6, and it is genuinely strong: the 27B dense runs from 8 GB VRAM up, the 35B MoE (3B active) is the coding pick at 24 GB. One line with a current llama.cpp era stack:

ollama pull qwen3.6:27b

If you would rather click than type, the free and open source local AI studio Locally Uncensored has the Qwen family in its one click model catalog, checks your memory before you download, and speaks to 12 local backends. The full setup walkthrough with quant tables lives here: How to Run Qwen 3.6 Locally, and the complete Qwen 3.8 hardware math is in Can You Run Qwen 3.8 Locally?

Why bother running it at home

Privacy: your prompts never leave the machine. Cost: open weights turn a metered bill into a one time hardware decision. Control: the model you benchmark today is the model you run next year, no silent upstream swaps. For the first time, the frontier launch everyone is hyping comes with a version of itself you will actually own, in the same week's drop.