DEV Community: Matt Macosko

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

Matt Macosko — Tue, 18 Aug 2026 18:30:09 +0000

20×

Less waiting per turn

1024

Gemma’s sliding window, in tokens

12/12

Eval tasks passed on a 550-token prompt

🧰The setup

I maintain claude-code-local, a repo for running coding agents against local models on a Mac. No cloud, no API key. The original approach pointed Claude Code at a local MLX server through a proxy. It works, and it is still in the repo. But Claude Code was designed for cloud models: its system prompt is tens of thousands of tokens, and parts of it change every turn. A local model pays for that twice, once prefilling a huge prompt, and again because a prompt whose head keeps changing defeats KV cache reuse completely.

So we built the obvious alternative. A small native engine, about 900 lines of Python on mlx-lm. Fixed 550-token system prompt, the same tools (bash, read, write, edit, glob, grep), and a KV cache that gets trimmed to the shared prefix each turn so only the new tokens are ever prefilled.

🐛The bug

Benchmarking surfaced something I did not expect. Short conversations were fast, exactly as designed, about a third of a second to the first token. But past a certain conversation length, every turn suddenly cost six and a half to seven seconds, as if the cache did not exist. It was not gradual. It was a cliff.

The cliff turned out to be Gemma’s sliding window attention. Gemma-family models give five out of every six layers a RotatingKVCache capped at the window size, 1024 tokens on Gemma 4. The moment your transcript outgrows the window, those rotating caches report themselves as untrimmable, and mlx-lm’s prompt cache reuse silently dies. Every turn re-prefills the entire transcript. The longer your session, the worse it gets, which means the failure lands exactly where caching matters most. There is no error, no warning, nothing. It just gets slow.

If you are building a Gemma-based agent on mlx-lm, check for this. You probably have it right now.

🔧The fix

Give every layer a plain KVCache instead. That sounds like it should change the model’s output, but it does not: the sliding window attention mask is what enforces the window. The cache type only decides what gets stored. We verified this the honest way, with greedy decoding, same conversation, stock caches versus plain caches, and the outputs were byte-identical on every turn.

The numbers, on a Gemma 4 31B (4-bit) with a 4,500-token conversation:

	time to first token
stock rotating cache	6.5 to 7.2 s
plain KV cache	0.36 s

That is roughly 20 times less waiting per turn, on the same model and the same MacBook. The trade is that KV memory now grows with the transcript instead of capping at the window, so the engine shows a live context meter, and an environment variable restores stock behavior if you would rather have the memory ceiling.

🧠But did removing the big harness make it dumber?

Fair question. Claude Code’s giant prompt exists for a reason, just not for a 31B model. We built a 12-task eval: create and run scripts, fix a failing test, rename across files, escape-heavy file content, precise edits, CSV work. All machine-checked by actually running the results, at temperature 0.

Qwen 3 Coder went 12 for 12 with the bare 550-token prompt. Gemma went 11 for 12, and its one failure was instructive: asked to write a file full of quotes and backslashes, it piped the content through shell echo, and sh’s echo silently collapsed the backslashes. A five-line prompt rule, never write file contents through the shell, always use the write and edit tools, took it to 12 for 12 with no regressions.

So no. For models this size, less harness turned out to be more capability, as long as the few rules you do include are aimed at failures you actually observed.

📦Where it all lives

Everything shipped today in the repo: the engine, the fix, the benchmark script so you can reproduce the numbers on your own machine, and the write-up. Existing Claude Code launchers are untouched, this is a second path and not a replacement. Credit where it is due, the prompt cache trim fix contributed in PR #46 is what made the deeper rotating cache problem visible at all.

Local AI on a Mac keeps surprising me. The models were already good.

🚰The gap has been in the plumbing, and the plumbing bugs are small, findable, and fixable.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

Matt Macosko — Fri, 07 Aug 2026 19:27:23 +0000

This morning I noticed my local coding agent answering way faster than it used to, and I couldn't explain why. I don't like speedups I can't explain, so we benchmarked it instead of guessing. What came out of that is the biggest single improvement my local setup has ever gotten, and a bug report that probably applies to your setup too if you run Gemma-family models on Apple Silicon.

The setup

I maintain claude-code-local, a repo for running coding agents against local models on a Mac — no cloud, no API key. The original approach pointed Claude Code (the CLI) at a local MLX server through a proxy. It works, and it's still in the repo. But Claude Code was designed for cloud models: its system prompt is tens of thousands of tokens and parts of it change every turn. A local model pays for that twice — once prefilling a huge prompt, and again because a prompt whose head keeps changing defeats KV-cache reuse completely.

So we built the obvious alternative: a small native engine, about 900 lines of Python on mlx-lm. Fixed ~550-token system prompt, the same tools (bash, read, write, edit, glob, grep), and a KV cache that gets trimmed to the shared prefix each turn so only the new tokens are ever prefilled.

The bug

Benchmarking the engine surfaced something I didn't expect. Short conversations were fast, exactly as designed — 0.3 seconds to first token. But past a certain conversation length, every turn suddenly cost 6.5 to 7.2 seconds, as if the cache didn't exist. It wasn't gradual. It was a cliff.

The cliff turned out to be Gemma's sliding-window attention. Gemma-family models give five out of every six layers a RotatingKVCache capped at the window size — 1024 tokens on Gemma 4. The moment your transcript outgrows the window, those rotating caches report themselves as untrimmable, and mlx-lm's prompt-cache reuse silently dies. Every turn re-prefills the entire transcript. The longer your session, the worse it gets — which means the failure lands exactly where caching matters most, and there's no error, no warning, nothing. It just gets slow.

If you're building a Gemma-based agent on mlx-lm, check for this. You probably have it right now.

The fix

Give every layer a plain KVCache instead. That sounds like it should change the model's output, but it doesn't: the sliding-window attention mask is what enforces the window. The cache type only decides what gets stored. We verified this the honest way — greedy decoding, same conversation, stock caches vs plain caches, and the outputs were byte-identical on every turn.

The numbers, on a Gemma 4 31B (4-bit) with a 4,500-token conversation:

	time to first token
stock rotating cache	6.5–7.2 s
plain KV cache	0.36 s

That's roughly 20× less waiting per turn, on the same model and the same MacBook. The trade is that KV memory now grows with the transcript instead of capping at the window, so the engine shows a live context meter, and an env var restores stock behavior if you'd rather have the memory ceiling.

But did removing the big harness make it dumber?

Fair question — Claude Code's giant prompt exists for a reason, just not for a 31B model. We built a 12-task eval (create-and-run scripts, fix a failing test, rename across files, escape-heavy file content, precise edits, CSV work — all machine-checked by actually running the results, temperature 0). Qwen 3 Coder went 12 for 12 with the bare 550-token prompt. Gemma went 11 for 12, and its one failure was instructive: asked to write a file full of quotes and backslashes, it piped the content through shell echo, and sh's echo silently collapsed the backslashes. A five-line prompt rule — never write file contents through the shell, always use the write/edit tools — took it to 12 for 12 with no regressions.

So no. For models this size, less harness turned out to be more capability, as long as the few rules you do include are aimed at failures you actually observed.

Where it all lives

Everything shipped today in the repo — the engine, the fix, the benchmark script (bench/agent_bench.py) so you can reproduce the numbers on your own machine, and the write-up. Existing Claude Code launchers are untouched; this is a second path, not a replacement. Credit where it's due: the prompt-cache trim fix contributed in PR #46 is what made the deeper rotating-cache problem visible at all.

Local AI on a Mac keeps surprising me. The models were already good. The gap has been in the plumbing — and the plumbing bugs are small, findable, and fixable.

matt

I wrote the missing Apple Silicon runtime for NVIDIA's Nemotron Omni

Matt Macosko — Thu, 06 Aug 2026 18:37:23 +0000

NVIDIA's Nemotron-3-Nano-Omni-30B-A3B is an open-weights model that sees, hears and reasons. There is already a 4-bit MLX quantization of it on Hugging Face, done by yayr. But as that model card says plainly, only the text backbone loads with standard MLX tooling:

The vision and audio towers require a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes.

Nobody had written that runtime. So the model could talk on a Mac, but it could not see or hear.

I wrote it. It is pure MLX: the vision tower, the audio tower, the processor, and the multimodal token splicing, all ported from NVIDIA's reference implementation.

Verified, not asserted

The thing I actually care about here is not that it runs. It is that I can prove it runs correctly.

Every component is tested against NVIDIA's PyTorch reference on the same inputs with the same weights, in fp32 on CPU so the comparison is honest. pytest tests/ — 23 of 23 passing.

Component	Compared against	Result
Audio tower	`transformers` ParakeetEncoder + NVIDIA `SoundProjection`	cos 0.99999130 (min/frame)
Audio frontend	log-mel features	max abs delta 8.1e-6
Vision tower	`nvidia/C-RADIOv4-H` via `trust_remote_code`	cos 0.99996227 (min/token)
Vision tower, MLX CPU stream	same	1.00000000 — graph-exact

A port that is almost right is worse than no port at all, because you spend weeks chasing quality problems that are really numerical drift in a tower you never checked. Writing the parity harness first was the single best decision in this project.

Speed and memory

Measured on an M5 Max MacBook Pro, running the 4-bit quantized language model with both towers in bf16:

Image:  67.7 tok/s · 22.1 GB peak
Audio:  147  tok/s · 21.0 GB peak
Text:   152  tok/s · 17.9 GB peak

Wifi off the whole time. Nothing leaves the machine.

That 22.1 GB peak on the image path is the number I would pay attention to if you are deciding whether to bother. It suggests this fits on a 32 GB Mac. I cannot verify that, because the M5 Max is the only machine I have. If you run it on something smaller I would genuinely like to hear what happens — that is the most useful thing anyone could send me right now.

Why bother doing this locally

The obvious question is why not just call an API. A few reasons that matter to me.

The model is open weights. Someone should be able to run open weights on their own hardware, and if the only path requires a vendor's cloud, the openness is partly decorative.

Apple Silicon is genuinely fast enough now. 67 tokens a second while reading an image, on a laptop, is not a compromise.

And there are workloads where the data cannot leave the building at all. I do work that touches NDA and compliance-sensitive material, and "it runs offline" is not a nice-to-have there, it is the requirement.

Try it

MIT licensed: https://github.com/nicedreamzapp/nemotron-omni-mlx

Credit to NVIDIA for publishing the weights, and to yayr for the 4-bit conversion.

NVIDIA Shipped a Model That Sees and Hears — It Just Didn’t Run on a Mac. So I Wrote the Missing Piece.

Matt Macosko — Tue, 28 Jul 2026 18:30:07 +0000

NVIDIA shipped a 30-billion-parameter model that can see, hear, and talk — and gave the weights away. The catch: the seeing and hearing parts didn’t run on a Mac. So I spent an afternoon writing the missing piece.

Here is the whole thing in thirty-five seconds — the model reading a real cart off my own store, on the laptop, with nothing leaving it.

Thirty-five seconds: the model reads a real cart off my own store, on the laptop, with nothing leaving it. The elapsed counter is the real measured latency.

Every couple of weeks I go looking for whatever new open-weight model just dropped, pull it onto my laptop, and see what it can actually do. Most of the time it’s a coding model, I run it against the one I already use, and my current favorite wins again. That’s a fine result. It’s just not much of a story.

This time I found something different.

What Nemotron Omni actually is

NVIDIA released Nemotron-3-Nano-Omni-30B-A3B — a “tri-modal” model, which is a fancy way of saying one brain with eyes and ears attached. You can hand it a picture, a sound file, or a video, and talk to it about what it saw or heard. It’s 30 billion parameters total, but only about 3 billion of them fire for any given word, which is why something this capable can run on a laptop at all. The weights are public.

Someone had already done the hard, unglamorous work of shrinking it down to a 4-bit MLX version that fits on Apple Silicon — that’s yayr over at mlx-community, and this project doesn’t exist without that upload. About 19 GB on disk. Ready to go.

Except for one line, buried in the model card:

The text backbone loads with standard MLX nemotron_h tooling. The vision and audio towers require a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes (e.g. the Evorix on-device engine).

Translated: the brain works on a Mac. The eyes and ears don’t. The weights for them are right there in the file — all the knowledge, sitting on your disk — but nothing I could find in the open Apple Silicon world knew how to run them. NVIDIA’s own code for those parts is written for their GPUs. On a Mac it’s a locked room with the key visible through the window.

I want to be precise about that parenthetical, because it matters: the card does name an engine, Evorix. I went looking for it — not on Hugging Face, not on GitHub, not anywhere I could find. So as far as I can tell it exists, but not somewhere you or I can go get it. Which leaves the same practical dead end: you download nineteen gigabytes of a model that sees and hears, and on a Mac you can only talk to it.

That’s the gap — no open runtime for the eyes and ears. Honestly, that’s the most interesting kind of thing to find. Not a benchmark to run. A thing that doesn’t work yet.

The part I keep having to relearn

My instinct was that this would take days. Three separate pieces, each one a real port: NVIDIA’s vision tower is a ViT with a custom patch generator, their audio side is a Conformer with Transformer-XL relative attention, and then there’s the glue that turns a photo into something the language model can read.

It took about half an hour.

Not because I’m fast — because I stopped doing it one piece at a time. I put a separate agent on each tower and let them run at the same time, each one checking its own work against NVIDIA’s original code as it went. I keep making the same mistake of estimating this stuff like it’s still 2024, and I keep getting corrected by my own laptop.

Proving it, instead of vibing it

Here’s the thing about porting a model: it’s very easy to write code that looks right, produces numbers, and is quietly wrong. The model doesn’t crash. It just gets a little dumber, and you never find out.

So neither tower got to claim victory on vibes. For each one, the test was: run NVIDIA’s original PyTorch code and my MLX version on the same input, with the same weights, and compare the actual numbers coming out.

Tower	What was compared	Result
Ears (audio)	Final audio embeddings, 5-second clip	cosine 0.99999
Eyes (vision)	Final image embeddings, 448px image	cosine 0.99996
Eyes, on CPU math	Same, without GPU shortcuts	1.00000000 — exact

That last row is the one I care about. Run it on the CPU, where the math is done precisely, and my port and NVIDIA’s are not close — they’re identical. The tiny gap on the GPU isn’t a bug in the port; it’s Metal taking shortcuts with float math for speed. Chasing that would be chasing the hardware.

The input processing got the same treatment — every image tile, every audio frame, every token id checked against NVIDIA’s reference. 14 tests, token sequences matching exactly, pixels off by less than a millionth.

The brain, meanwhile, needed no porting at all — MLX already understood it. Someone had put nemotron_h into mlx-lm before I ever showed up.

So does it actually work

This is the only part that matters, so here’s the first thing I pointed it at once the pieces were connected. Not a test image — a screenshot off my own phone, of my own store, with a cart full of my own products.

I asked: “What website is this and what is in the cart?”

This is the Divine Tribe website. The cart contains a Gen 2 DC Ceramic Rebuildable Dry Herb Heater, a Replacement Heater Cup, a Wireless Dock Station, 72% Hemp 28% Silk Men’s Boxers, and a Quest Lightning Diffuser Kit.

Every product, correct. When I asked for prices it read all five to the cent — $37.36, $13.12, $80.80, $33.32, $38.67 — and started adding them up. Nine seconds, on a laptop, with nothing leaving it.

Then the ears. I generated a line of speech and handed it the wav:

The Divine Tribe vaporizer ships from Humboldt County, California, and this model is running entirely on a MacBook with no Internet connection.

Word for word, punctuation and all — it even got “Humboldt” right.

The speeds, for anyone keeping score:

Mode	Speed	Memory
Text only	152 tok/s	17.9 GB
With an image	67.7 tok/s	22.1 GB
With audio	147 tok/s	21.0 GB

The whole thing lives in about 22 GB at its hungriest. That fits on a Mac you can buy today.

And to be clear about what that means: nothing about that cart screenshot or that audio clip left my desk. The model makes no network calls at all — there’s no API key, no meter running, no terms of service, and no company on the other end deciding whether I’m allowed to keep doing this tomorrow.

Two things that fell out along the way

Neither of these was the goal, and both are probably more useful than anything else here.

NVIDIA’s reference code produces NaN on batched audio. If you feed it two clips of different lengths at once, the shorter one comes back as garbage — not an error, just silent NaN poisoning that spreads through the layers. It’s a masking detail: fully-padded rows go to negative infinity, softmax turns that into NaN, and the NaN travels. My port masks differently and stays finite. I want to be careful here — this is one specific path, and I could be wrong about how much it matters in NVIDIA’s own pipeline. But it reproduces on their code, not just mine.

The vision tower doesn’t normalize its own input. There’s a normalization layer sitting right there in the checkpoint, and it’s dead weight — NVIDIA switches it off and expects whatever calls it to do that job. Feed it raw pixels like a reasonable person would and you get plausible-looking garbage, silently. This one cost real time and it’s the trap anyone else attempting this will hit.

And a third, for the truly nerdy: several settings in the config file are lies. Not maliciously — they’re just vestigial, left over from an earlier design, and the live code ignores them completely. If you build from the config instead of tracing what actually runs, you’ll produce something that looks correct and isn’t. I only caught it by following the real code path line by line.

So who does this actually help?

Fair question, and I want to answer it honestly instead of waving at the future.

The narrow answer: it saves the next person about a week. Right now, anyone with a Mac who downloads this model reads that line on the model card — “requires a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes” — and that’s the end of the road. It’s not a warning, it’s a wall. It means go build it yourself, and most people, reasonably, close the tab. Now they don’t have to. Clone it, run it, done. And the three traps I hit are written down, because the normalization one in particular would cost someone a full day of wondering why their model got quietly dumber instead of visibly broken. That’s the entire contribution: one person spent the afternoon so nobody else has to spend the week.

I’m not going to pretend that’s a huge number of people. Might be a few hundred. Might be twelve. That’s fine — twelve people not wasting a week each is still worth an afternoon.

The wider answer is the one I actually care about. I do private-AI work for firms that handle other people’s confidential material — lawyers, medical practices, accountants. The whole pitch is that the machine doing the reading is the machine on your desk, because a federal court already ruled that work you hand to a public AI isn’t privileged.

Until now, “on your desk” meant text only. If a client sends a photograph of a contract, or a recorded call, or a scan — the private option had nothing to say. You either sent it to somebody’s cloud and lost the privilege, or you did it by hand.

That changed today, on my laptop. A model that can look at the scanned page and listen to the recording, on the machine sitting in front of them, is not a small difference for those people. It’s the difference between a tool they can use and a tool they legally can’t.

And the honest third reason: the gap between “the weights are public” and “you can actually use this” is where most open AI quietly dies. Everyone celebrates the release. Far fewer people do the boring work of making the thing run somewhere real. That gap is usually not a research problem — it’s a few hundred lines nobody got around to writing. This one was maybe 90 KB of Python and an afternoon.

That’s the part I’d like more people to see. Not that I did something clever — I didn’t, I transcribed NVIDIA’s own math into a different framework and checked my work. But the wall between an open model and a working model is thinner than it looks, and it stays up mostly because everyone assumes someone else will knock it down.

The code is on GitHub, MIT, with every parity test in it. Don’t take my word for any number above — clone it and run the tests on your own Mac.

Credit where it’s due

NVIDIA built the model and released the weights and reference code openly. None of this happens otherwise.
yayr at mlx-community did the 4-bit MLX quantization I built on top of. Go give that upload a like — it deserves more than the zero it has.
Apple’s MLX team — and whoever added nemotron_h to mlx-lm, which is why the brain needed nothing from me at all.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

The Day Local AI Caught the Cloud: ds4, DeepSeek V4 Flash, and What Just Changed for Devs

Matt Macosko — Thu, 23 Jul 2026 18:30:07 +0000

If you write code for a living and you’ve been watching the local-AI space, May 9, 2026 is the date to circle. Salvatore Sanfilippo (yes, the guy who wrote Redis) shipped ds4 — a few thousand lines of hand-written C with Metal compute kernels, built for exactly one model: DeepSeek V4 Flash.

I ran the same prompt through three engines on the same 128 GB MacBook Pro:

DeepSeek V4 Flash via ds4 — fully local, off-cloud
Cloud Claude through my Max plan
Gemma 4 31B via MLX, also local

Local DeepSeek beat cloud Claude on wall-clock time. That sentence used to be science fiction.

▶ Watch the companion video — three engines, one prompt, three completely different aurora animations rendered in real time on the same machine.

The benchmark, for people who don’t want filler

Engine	Time	Output	Where it ran
DeepSeek V4 Flash (`ds4` local)	103 s	3,259 tokens	Apple Silicon GPU
Cloud Claude (Max plan)	192 s	~3,500 tokens	Anthropic data center
Gemma 4 31B (MLX local)	131 s	1,992 tokens	Apple Silicon GPU

The prompt was a single creative HTML task: “Build an animated northern lights scene — single file, vanilla JS, mountains, pine trees, twinkling stars, flowing aurora bands.”

Each engine produced a completely different aurora. None of them hit the network during inference. (Yes, I checked with lsof. Yes, this is the same lsof audit pattern from the AirGap NDA piece.)

Three architectural decisions in `ds4` worth understanding

This is the part that matters if you’re a developer thinking about local AI infrastructure.

1. Asymmetric 2-bit quantization (only where quality is forgiving)

The naive approach to quantization treats every weight the same. ds4 doesn’t. Only the routed Mixture-of-Experts experts get compressed to 2-bit (specifically IQ2_XXS for up/gate, Q2_K for down). Every quality-critical path — shared experts, attention projections, routing, output head — stays at higher precision (Q8 or full).

Those routed experts are about 90% of the weight footprint. The other 10% is where small precision losses cause big accuracy losses. Quantize the 90%, leave the 10%, and you get an 81 GB file that still calls tools cleanly and writes coherent code.

This is the kind of tradeoff that only makes sense if you’ve stared at a specific model’s loss landscape long enough to know which weights tolerate compression. It’s a model-specific engineering decision dressed as a quantization recipe.

2. KV cache moved to disk (in 2026 SSDs are fast enough)

The “KV cache must live in RAM” assumption is from 2023. Modern Apple SSDs do 5+ GB/s sequential reads. ds4 writes session state to disk and reuses it across runs, keyed by SHA1 of token IDs.

The practical effect: when Claude Code sends its 25k-token system prompt, that prefill happens exactly once, ever. Every subsequent session — including totally different agent runs that happen to share that prefix — reads from disk in milliseconds instead of recomputing from token zero.

If you’ve used long-context models locally, you know prefill is the slowest thing in the loop. ds4 makes it free after the first hit. That’s the kind of “small change, huge implication” move that took years to normalize. (See also: the disk-KV section in the ds4 README.)

3. Pure Metal, not CUDA-with-a-shim

There’s no PyTorch, no TensorFlow, no llama.cpp wrapper layer in the hot path. The compute kernels under metal/*.metal are written specifically for this one model on this one architecture. The acknowledgments thank llama.cpp and GGML — ds4 borrows quant layouts and select kernels — but it’s not a fork.

This narrowness is the point. Generic frameworks pay a tax for being generic. When you commit to one model on one chip, you can hand-tune away that tax. ~27 tok/s on an M3 Max 128 GB. ~32 tok/s on M5 Max. For agent loops on a laptop, that’s plenty.

Why this matters for compliance-sensitive devs

The same week, I’m still maintaining AirGap AI — a wi-fi-off, lsof-audited workflow for analyzing privileged documents (NDAs, client files, PHI, etc.) on a laptop with no outbound connections. Until last week, that was a Llama 3.3 70B story. The capability ceiling was real.

ds4 raises that ceiling materially:

1M-token context — entire codebases, full deposition transcripts, complete contract sets, all in-memory in a single conversation
Quasi-frontier reasoning — if you’ve used Claude Sonnet or Opus, DeepSeek V4 Flash sits in the same neighborhood for most agentic tasks
Tool calling that works — Antirez tested it under coding agents (opencode, Pi, Claude Code) and the tool calls land reliably

For law firms, medical practices, and compliance-bound shops, the math just changed. You don’t have to choose between “frontier-grade reasoning” and “data never leaves the building.” The hardware exists, the engine exists, the model exists, and the integration with Claude Code exists.

(If you’re trying to get a bar-association-defensible AI workflow off the ground, the AirGap landing page is where I keep my notes. The ds4 stack is going in there next week.)

How to actually run it

The full stack, all open-source:

# 1. Build the engine (Apple Silicon with Metal)
git clone https://github.com/antirez/ds4
cd ds4 && make

# 2. Pull the q2 weights (~81 GB)
./download_model.sh q2

# 3. Boot the local Anthropic-compatible server
./ds4-server --ctx 200000 --kv-disk-dir ~/Library/Caches/ds4-kv \
             --kv-disk-space-mb 16384

# 4. Point Claude Code at it
ANTHROPIC_BASE_URL=http://127.0.0.1:8000 \
ANTHROPIC_AUTH_TOKEN=dsv4-local \
ANTHROPIC_MODEL=deepseek-v4-flash \
claude

Or just clone nicedreamzapp/claude-code-local — DeepSeek V4 Flash is now the fourth fighter in the lineup, with a claude-ds4 wrapper that handles all of the above for you.

What this slots into

This isn’t a one-off. It’s the next click in a longer arc I’ve been writing about:

Three Generations of Running Claude Code Locally on a MacBook — What I Actually Learned — the long path from “barely works” to “actually replaces my cloud usage”
Cloud AI Coding Costs Keep Climbing — How to Pay $0 and Still Use Claude Code — the economic angle, before ds4
Pulling 10x My Subscription Value Out of Claude — what the cloud math actually looks like
What It’s Actually Like to Code By Voice — With the AI Replying In My Own Cloned Voice — the voice loop these models now plug into
Your Medical Practice Is Probably Using Cloud AI on PHI Right Now — why on-device matters for healthcare
If Your Law Firm Is Using Cloud AI on Client Files, You Probably Have a Problem — the legal angle
A Field Guide to Ambient Computing — the bigger frame this all sits inside

ds4 is the engine that finally makes the local-first version of all of those usable for production work. The local agent doesn’t have to pick which workload it’s good at anymore.

Where to follow

🛠️ github.com/nicedreamzapp/claude-code-local — the lineup, launchers, and benchmarks
🐳 github.com/antirez/ds4 — the engine itself
🌿 marijuanaunion.com — the broader writing on local AI, voice, and ambient computing
🔒 nicedreamzwholesale.com/airgap — the compliance-grade workflow notes
💬 Discord — NiceDreamzApps server

May 9, 2026. The day a single C file caught up to the data centers.

This is the technical companion to the headline piece on Marijuana Union. Companion video: youtu.be/7l8-s8xkpms. For local-AI consulting on compliance-sensitive workloads, see AirGap AI.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

The IRS Said Yes — and What That Does and Doesn’t Mean

Matt Macosko — Thu, 16 Jul 2026 06:10:44 +0000

Back on June 3rd I published a piece here saying the Cannabis Device Safety Institute had filed its federal application, and I made a point of leaning on that word — application. Filed, not granted. Pending, not approved. Not a tax-exempt charity, contributions not deductible, and we’d say so plainly every time, because saying it any other way would be exactly the kind of thing this institute exists to push against.

So I owe you the update in the same register.

The IRS granted it. The determination letter is dated June 29, 2026. CDSI is exempt from federal income tax under Section 501(c)(3), classified as a public charity under Section 509(a)(2) — not a private foundation — with the exemption effective April 27, 2026, retroactive to the day the articles were filed. Contributions are deductible under Section 170.

That’s the whole announcement. Now let me do the part I think matters more, which is being precise about what it does and doesn’t mean.

What it doesn’t mean

It is not an endorsement. Recognition of exemption is a determination about tax status. The IRS did not review our methodology, did not evaluate our off-gas protocol, and has no opinion whatsoever about whether ceramic donut atomizers off-gas at 650°F. No agency has blessed CDSI’s findings, because CDSI hasn’t published findings yet. If you ever see me imply otherwise, call me on it.

It doesn’t make CDSI a regulator. We’re a private nonprofit. We have no authority over anyone, we can’t compel any manufacturer to do anything, and we’re not seeking a government designation. The model is UL, ASTM, the NFPA Research Foundation — bodies that earned standing by being useful and rigorous, not by being appointed.

It doesn’t mean we have a lab — yet. Right now CDSI pays accredited independent labs and publishes what comes back. Pay the lab, not pay to pass. But the goal was never to outsource forever: the plan is for this institute to build and run its own testing bench, because the body that writes the methodology should eventually be able to execute it too — and the determination letter is exactly what makes that fundable. Foundation grants and tax-deductible donations can now go toward standing up a lab of our own. If and when that lab exists, nothing about the discipline changes — open methodology, public reports, every conflict on the cover page.

It doesn’t finish the paperwork. California still treats CDSI as a taxable corporation until a separate state filing goes through. That one’s in an envelope, not a press release.

About the timeline, honestly

We filed the full Form 1023 on June 3. The determination is dated June 29 — 26 days later.

The IRS publishes exactly one number about how long this takes: “We issue 80% of Form 1023 application determinations within 191 days.” That’s from their own “Where’s my application” page. So: 26 days, against a published benchmark of 191, on the long form rather than the 1023-EZ shortcut, with no request for expedited handling.

I want to be careful with that number, because it would be very easy to turn it into something it isn’t.

I can’t tell you it’s a record. Nobody can tell you that, about any organization. The IRS’s public files record determination dates by month only — no day — and carry no application-submitted date at all. The interval literally cannot be computed from public data, including the IRS’s own. So anyone claiming a record in this category is claiming something no dataset could check, and I’d rather not be that guy.

I also can’t tell you the speed proves the application was good. That’s the flattering read, and I don’t think it survives scrutiny. The IRS has fast-track lanes for straightforward cases, and how big those lanes are isn’t published. The six days between answering their follow-up letter and getting the determination is, as far as I can tell, just the IRS’s own internal rule about how fast a specialist has to close a case once you respond. And the biggest variable — when the application got assigned to a human at all — was completely outside my control and I can’t explain why it happened when it did.

What I can say is what we actually did, and let you decide if any of it mattered:

We paid $600 for the long form instead of $275 for the EZ, on purpose, because the EZ doesn’t have room to explain why a fee-for-service testing subsidiary serves a public mission — and filings like ours get bounced back to the long form anyway, months later. The slow-looking choice was the fast one.

We disclosed the conflicts instead of burying them. The founder of a cannabis hardware standards body builds cannabis hardware. That’s on the record, in the conflict of interest policy, in the state charity filing, and it’ll be on the cover page of every report we publish.

We went through this site and cut every claim we couldn’t prove — before we filed, not after. No “first,” no “leading,” no partnerships that hadn’t happened. Reviewers read your website. There was nothing to pick at because we’d already picked at it ourselves.

And when the IRS wrote asking for more information, we answered within hours of opening the envelope instead of sitting on the 28 days we had.

None of that is clever. It’s just doing the boring things in the right order.

What it actually changes

Donations to CDSI are now tax-deductible. That’s real — it means a foundation can fund hardware testing, and an individual who cares about this can help and take the deduction.

CDSI is registered with the federal government and eligible for federal grants — that happened back on June 23, with the SAM.gov registration going active.

Put those together and here’s the thing I care about: the money to characterize this hardware can now exist. For fourteen years the reason nobody tested the device was that no one would pay for it. In 2016 I paid a lab out of pocket to run an off-gas test on a concentrate vaporizer because there was no funding source in the world for that work. That’s still the founding artifact of this institute, and it’s still the reason it exists.

Now there’s a vehicle that can receive that money. That’s what a determination letter is. Not a trophy — plumbing.

Where this goes

The next thing I want to make the case for is bigger than safety, and I’ll write it properly soon.

It’s this: you cannot honestly answer whether inhaled cannabis helps someone until you know what the device contributed to the dose. Heat a vaporizer and it puts its own chemistry into the same stream carrying the cannabis. So every measured outcome — a contaminant, a symptom, a relief — has two possible sources: the material and the machine. That’s not noise you can fix with more study subjects. It’s an attribution problem, and no sample size touches it.

Every other measurement science solved this a century ago. The chemist runs a blank before running samples. Nobody doses a patient through an uncharacterized nebulizer. Cannabis research does the equivalent constantly, not out of carelessness, but because no one was ever responsible for the device.

Which means hardware characterization isn’t adjacent to the medical question. It’s upstream of it. First you characterize the device. Then you have a device you can trust as an instrument. Then — and only then — you can ask what the medicine does to a person and believe the answer.

It all starts with the devices. Not because the devices matter most. The person matters most. But the device is the part you have to understand first in order to understand any of the rest of it honestly.

That’s the work. The letter just means we can afford to do it.

The Cannabis Device Safety Institute is a California nonprofit public benefit corporation, recognized by the IRS as exempt under IRC § 501(c)(3) and classified as a public charity under § 509(a)(2) (determination letter dated June 29, 2026; EIN 42-2429365). Recognition of exemption is a determination of federal tax status and is not an endorsement of the Institute or its findings by the IRS or any government agency. cdsi.click

Originally published at Marijuana Union. The Cannabis Device Safety Institute is an independent 501(c)(3) nonprofit standards body for cannabis consumption hardware — open methodology, public reports, every conflict of interest on the cover page. Methodology, papers, and the public record: cdsi.click.

M5 Max + 128GB = a 30B AI Coding Agent Running Locally. Wi-Fi Off.

Matt Macosko — Mon, 29 Jun 2026 17:18:47 +0000

The M5 Max MacBook Pro with 128 GB of unified memory is the first laptop that can hold a frontier-class coding agent entirely in RAM. No GPU rack. No cloud. No subscription.

That clip up top isn’t a render. That’s Qwen 3 Coder — 30 billion parameters, 8-bit MLX — running on this MacBook with the Wi-Fi off. Around 55 tokens per second. Total cost to keep running it: zero.

The thing that matters more than the spec sheet is what it actually unlocks.

Why the M5 Max changes the math

Until now, running a 30B+ parameter model meant a GPU rack — or paying a cloud API per token. The M5 Max changes that:

128 GB unified memory. The entire model lives in fast RAM. No GPU offload, no quantization tricks past 8-bit. The CPU and “GPU” share the same memory, so there’s no copy step between them.
Mixture-of-experts plays perfectly with Apple Silicon. Qwen 3 Coder is 30B total but only 3B active per token. That’s a math problem the M5 Max’s memory bandwidth eats for breakfast.
MLX runs at near-CUDA speed. Apple’s native ML framework hits ~55 tok/s on the 8-bit quant. No CUDA tax, no Nvidia driver politics, no $40,000 GPU bill.
It’s a regular store-bought laptop. No GPU rack. No data center. No cloud bill. You can run it on a plane.

This wasn’t possible on a laptop a year ago. It is now.

What you can do with it now

Read a legal contract — and have it never leave your machine.

Most AI tools pipe your document to a server somewhere. With this setup, the bytes don’t leave the laptop. NDAs, supplier agreements, employment contracts — review them at your kitchen table without uploading them to anyone.

Write production code in a couple of seconds.

The video shows it: real Python function, real Qwen output, no edits. The agent’s tool-calling is good enough to drop into Claude Code’s loop, where it’ll edit files, run shell commands, and iterate. It’s plenty for everything from one-off scripts to refactoring real production code.

Analyze patient charts without a HIPAA violation.

For doctors, therapists, intake clinics — anything with PHI on it — local-only AI isn’t a nice-to-have, it’s the only legal option. Same model, same speed, zero bytes leaving the device.

Build agents that don’t charge you per call.

This is the one most people sleep on. Pay-per-token cloud APIs make agents expensive to leave running. Once the model is local, you can let an agent loop overnight, hit it with thousands of requests, kick off a watcher that scans your inbox every two minutes — and the cost stays at zero.

The full stack

Here’s the receipt:

Hardware: M5 Max MacBook Pro, 128 GB unified memory.
Model: Qwen3-Coder-30B-A3B-Instruct-MLX-8bit — about 30 GB on disk. Mixture-of-experts, ~3B params active per token.
Server: A small Python proxy at localhost:4000 that speaks the Anthropic Messages API, so the Claude Code CLI thinks it’s talking to the cloud — except it’s talking to a hard drive.
Total monthly cost: $0 once it’s downloaded.

That’s it. No Docker, no Kubernetes, no VPS. Just a laptop on a desk.

The performance, honestly

The local-AI space is full of overclaims, so the straight numbers:

55 tokens per second on a real coding task. Sustained, not peak.
Two seconds to write a working find_median() function. Three to four seconds for most refactors.
Tool-calling reliability is good enough for the Claude Code agentic loop. Not as locked-in as Sonnet 4.6, but plenty for getting work done.
What it’s not: a Sonnet replacement for nuanced reasoning, long contexts, or really tricky debugging. For day-to-day code agent work, it more than holds its own.

Why the offline part matters

The reason “Wi-Fi off” keeps coming back in the demo isn’t a gimmick. It’s the whole thesis.

If a tool needs the internet, three things are true:

Someone else can read what you sent.
Someone else can charge you for it.
Someone else can take it away.

If the same tool runs locally, none of those are true. That’s a different category of software. Not better at every task — but yours.

Benchmarks — actually run, not cited

Big claims need numbers. So here’s what Qwen 3 Coder 30B-A3B (8-bit MLX) actually scores on this MacBook, run end-to-end against the local localhost:4000 server. Every problem solved by the model, executed in a Python subprocess, scored pass/fail. Pass@1, temperature=0, single sample per problem.

Benchmark	N	Pass@1	Notes
HumanEval	164/164 (full)	81.7%	Python function-completion classic. Saturated benchmark; modern coding models cluster 75–95%. 14 min total wall-clock.
MBPP (sanitized)	168/427 (sampled)	83.3%	Mostly Basic Python Problems. Pass rate was stable since n=120; a few outlier tasks induce very long model responses, so I cut off at 168.

Both runs used pass@1, temperature=0, 10s execution timeout, on the local 8-bit MLX quantization. No retries. No best-of-N tricks. Single sample per problem.

For context — what the bigger sibling scores on harder benchmarks

The Qwen team didn’t publish HumanEval/MBPP for any Qwen3-Coder variant — they consider those benchmarks saturated. Their official benchmarks are agentic, and they ran them on the flagship Qwen3-Coder-480B-A35B-Instruct (the bigger sibling, ~16× the active params of the 30B-A3B running on this laptop). For context — here’s what the flagship 480B scores on those harder agentic benchmarks compared to the major closed models:

Agentic Benchmark	Qwen3-Coder 480B	Claude Sonnet 4	GPT-4.1	DeepSeek-V3
SWE-bench Verified (500-turn)	69.6	70.4	—	—
Terminal-Bench	37.5	35.5	25.3	2.5
BFCL-v3 (function calling)	68.7	73.3	62.9	64.7
Aider-Polyglot	61.8	56.4	52.4	56.9
WebArena	49.9	51.1	44.3	40.0

Source: Qwen team’s official blog. The 30B-A3B running on this MacBook is a smaller sibling of the 480B — it trades absolute peak agentic ceiling for fitting in 30 GB and running 24/7 on local hardware. For most coding tasks people actually do in a day, HumanEval/MBPP-class accuracy matters more than the SWE-bench top-line, and on those it sits where it should: useful, fast, local.

Where this is heading

The next year of the AI conversation isn’t going to be “which model is smartest.” It’s going to be “which workloads belong on your machine, and which belong on someone else’s.”

Compliance-bound work — legal, medical, financial — is going to move local fast. Code-agent loops will follow because the math (per-call cost vs. zero) is brutal. The M5 Max with 128 GB of unified memory is the laptop that lets that happen.

Try it yourself

The launchers are open source on GitHub: nicedreamzapp/claude-code-local. The README walks through downloading the model and pointing Claude Code at the local server.

For law firms, medical practices, and accountants that want help getting this running on their own hardware — that’s what AirGap is. 14-day pilot, fixed scope, the data never leaves your machines.

— matt

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

4 AI Models Built the Same Game on One Laptop — and a Local One Beat the Cloud

Matt Macosko — Tue, 23 Jun 2026 05:34:34 +0000

Four AI models. One job: build a playable game. One laptop. I gave four different AI models the exact same prompt — build a complete, playable Asteroids game in a single HTML file — and then I actually played what each one built. Three ran entirely on my laptop. One was the cloud, as the benchmark to beat. Here is the whole thing in under a minute.

I have been doing a lot of these little head-to-heads lately, and I wanted one where the test was real — not a riddle or a trick question, but an actual build. Give every model the same spec, let it write the whole thing in one shot, then open the file and see if the game actually plays. No cherry-picking, no fixing it up afterward. Either it runs or it doesn't.

The setup

Everything ran on a MacBook Pro M5 Max with 128 GB of memory. Three of the four models ran 100% local — no internet, nothing leaving the machine — through MLX and llama.cpp. The fourth was Cloud Claude Opus, sitting in a data center, there to set the bar.

Model	Where	Time	Speed
DeepSeek V4 Flash 284B	Local	300s	31 tok/s
Cloud Claude Opus	Cloud	48s	115 tok/s
Qwen3-Coder 30B	Local	33s	95 tok/s
Gemma 4 31B	Local	145s	26 tok/s

The prompt was identical for all of them: a single self-contained HTML file, vanilla JavaScript, no libraries — a ship that rotates and thrusts, bullets, asteroids that split when you shoot them, screen wrap-around, score, lives, and a game-over screen. One shot.

What happened

All four wrote a real, working game. That part genuinely surprised me — a few years ago, "write me a complete arcade game in one file" was not something you handed to a model running on a laptop and expected to play afterward.

DeepSeek built the most polished one. A glowing ship with a thrust trail, a starfield background, asteroids that actually broke apart, hearts for lives. It took the longest, but it played beautifully. Cloud Claude was fast and clean — lives, a level counter, even on-screen control hints. Qwen3-Coder 30B was the speed demon, turning out a solid game in 33 seconds. And Gemma delivered a tidy, working game of its own.

The verdict

The winner, to my eye, was DeepSeek — and the part that sticks with me is that it was running locally, on my desk, and it built a better game than the cloud model. Cloud Claude was excellent and much faster, but on the actual finished product, the local 284B model edged it out.

The bigger point is the one I keep landing on in everything I write here. Three of these four ran completely offline, for free, on one machine. No subscription, no metered tokens, nothing leaving the laptop. The cloud is still faster, and for a lot of work that speed matters. But "you have to use the cloud or it won't be any good" is just not true anymore. Local AI is catching up, and on this particular test it pulled ahead.

Run local models yourself (free)

If you want to try this, the abliterated MLX models I have converted for Apple Silicon are all free to pull:

My Hugging Face — abliterated MLX models for Apple Silicon
claude-code-local — run Claude Code against a local model, no API key.

The narration in the video is local text-to-speech, and the whole comparison was made on one MacBook. The only thing that touched the cloud was the Cloud Claude baseline.

Originally published at Nice Dreamz Wholesale. I convert abliterated MLX models for Apple Silicon — all free at my Hugging Face.

Eight local AI agents on a Mac mini — and the product I'm building from them

Matt Macosko — Tue, 26 May 2026 19:04:03 +0000

A case study went around recently: a lawyer had wired up 66 AI agents on a Mac mini for his own firm — every one running locally, nothing touching a cloud API — and was looking for a commercial partner before releasing it as open source.

I read that and realized I had been building the same shape of thing for my own small business for the last year. Eight ambient agents, all running on Apple Silicon, none of them touching cloud APIs. I had not productized any of them. I had also not gotten paid for any of them.

This post is about what I'm doing about that.

The asset base

I shipped a repo called claude-code-local. It's an MLX server that wraps three local language models (Gemma 4 31B, Llama 3.3 70B, Qwen 3.5 122B MoE) behind an OpenAI-compatible API. The setup script picks the right model for your hardware, downloads it, and puts a launcher on your Desktop. Three commands and you're running a 31-billion-parameter language model on your MacBook.

It has 2,689 stars and 516 forks as of this writing. License is MIT. The whole thing is at github.com/nicedreamzapp/claude-code-local.

That's the engine. What I had not built was the funnel.

The gap that ate me for a month

Stars are people who think "I would love this." Forks are people who started doing the work. Neither converts to revenue.

Reading that case study, what jumped out was: a lawyer with the technical chops to wire 66 agents could not productize the result. He has the buyer relationships (he is the buyer); I have the go-to-market background (I've sold consumer hardware direct to customers for years). The intersection nobody is shipping is "Mac mini with this stack pre-installed, delivered to your law firm."

So I sat down and mapped what was actually missing.

The market gap, in numbers

I researched the on-device AI market for May 2026. Here is what I found:

Free OSS (Ollama, LM Studio, Jan, Atomic Bot, my own repo): saturated, no money flowing
Paid Mac App Store apps (Private LLM, Enclave AI, Local LLM): $10-30 one-time, real revenue for bootstrapped 2-person teams
Cloud SaaS with zero-data-retention (Spellbook, CoCounsel): $69-$149/mo per seat, cloud-not-local
Enterprise legal AI (Harvey): $1,200+/seat/month
AI-native law firms (Manifest, Avantia, General Legal): not selling tools, they ARE the firm

Two things stand out:

The gap between "free OSS" and "$1,200/seat SaaS" is not being filled by subscription products. Private LLM's App Store reviews explicitly call out "no subscription" as the reason buyers picked them. The privacy buyer rejects recurring billing for privacy software. This is not opinion; it is in their reviews.
The Mac mini hardware bundle for privileged work is being built privately but not productized. Nobody is shipping it as a SKU.

That is the gap. That is what I am putting a product against.

The ladder

I'm skipping the standard SaaS playbook because the market is telling me to. The privacy buyer rejects recurring billing — so the spine is one-time purchases, not subscriptions:

Rung	Price	Audience
Free repo	$0	Existing OSS audience, top of funnel
AirGap Box	$2,995 (Base) / $3,995 (Pro)	Small firms wanting sovereignty without DIY
Foundation consulting	scoped	Firms ready for deeper, white-glove deployment

The free repo gets you curious. The Box converts the firms who would rather pay $3k than learn MLX. Foundation is for the firms that want it installed, tuned, and documented for their compliance counsel.

The three core agents

Three agents that drop on top of any OpenAI-compatible local LLM server (the free claude-code-local repo by default, but they work with Ollama and others too) — and ship pre-installed on the Box:

Folder Watcher — drop a PDF, text, or Markdown file into ~/AirGap-Inbox/. Within 30 seconds, a structured Markdown summary appears in _summaries/. Uses macOS's textutil and mdimport for PDFs and docx without external dependencies.
Daily Briefing — every morning at 7:00 a LaunchAgent reads the folders you list in a config file and writes a one-page digest of what changed in the last 24 hours.
Local Q&A — a CLI command airgap ask "your question" that answers from a single document you point it at, fully offline. Folder-wide indexing and citations across many files is on the roadmap — not something I'd oversell today.

The whole set is ~300 lines of Python. The installer is a .command file you double-click. Total install time: under 60 seconds on the right hardware.

I deliberately built them with zero authentication required. No IMAP credentials, no OAuth dance, no API keys — the install works for everyone on day one. The agents that need credentials (email drafting, Reddit lurking, calendar parsing) run on my own machines and ship in the AirGap Box, where there's a setup call to wire them up properly.

What's in the AirGap Box

The same stack on a Mac mini that arrives at your office. Pre-installed: claude-code-local MLX server, the three core agents, two more that need credentials (email drafter, prompt library), a default-blocked firewall, and a printed compliance memo template.

Base ($2,995): Mac mini M4 16GB, Gemma 4 31B preloaded.
Pro ($3,995): Mac mini M4 Pro 24GB, Llama 3.3 70B preloaded.

Both include a 90-minute Zoom setup call and 30 days of email support. After that, you have a working private-AI workstation. We do not phone home. We do not see your data.

COGS on the base unit is around $940. Margin around 69%. Cash-flow safe because Stripe charges at purchase; hardware ordered after.

Validation before scale

I am not ordering inventory on speculation. The plan is:

Day 0: launch publicly, with the Box gated by a waitlist (no checkout yet)
Day 7: first review — Box waitlist signups
Day 14: hard gate on Box — 20+ verified emails or we revisit positioning
Day 30: full P&L review, decide whether to scale

If the Box waitlist hits 20 in 14 days, I order the first 3 Mac minis. If it hits 5, I have not validated the rung; I revisit positioning before spending hardware money.

This is the part the grifter posts skip. "I made $14,200/month in 72 hours" is not a thing that happens to honest businesses. What happens to honest businesses is "I opened a waitlist, watched signups for two weeks, decided whether to order inventory based on actual demand, and reported the real numbers."

Honest year-1 range

Month	Pessimistic	Realistic	Stretch
1	$0	$400	$2,000
3	$200	$1,500	$6,000
6	$800	$3,500	$12,000
12	$1,500	$6,000-8,000	$20,000+

To hit the grifter's $14k/month claim, every rung needs to perform near the top of its range AND consulting needs to land. That happens in months 12-18 with discipline, not in 72 hours.

What I will publish honestly

Weekly waitlist + signup numbers (real, not vanity)
The first refund (when it happens) and why
The first Box install case study (with the firm's permission)
The Box waitlist → order conversion, as it happens

If you want to see whether this business works, follow along on github.com/nicedreamzapp/claude-code-local and the AirGap landing pages. I'll post the numbers as they happen.

Why I'm posting this

Because the next person to read a case study like that and think "I want this" deserves to find a productized version, not another Hacker News thread about MLX setup. And because every honest version of this story I publish is also a receipt — for me, that I built something real, and for the next builder, that the playbook works.

If you have feedback on the pricing, the positioning, or the agents, leave a comment. I read everything.

I'm building AirGap — pre-configured local AI on a Mac mini for firms handling private work. Join the AirGap Box waitlist, grab the free open-source stack, or see AirGap consulting for compliance-sensitive firms (law, medical, finance).

HumanEval on a MacBook — 81.7% pass@1, Wi-Fi off

Matt Macosko — Wed, 29 Apr 2026 18:11:49 +0000

The M5 Max MacBook Pro with 128 GB of unified memory is the first laptop that can hold a frontier-class coding agent entirely in RAM. No GPU rack. No cloud. No subscription.

I just ran HumanEval on it. Wi-Fi off the entire run.

81.7% pass@1 on the full 164-problem benchmark
Qwen 3 Coder 30B-A3B-Instruct (8-bit MLX)
14 minutes wall-clock, $0/month after the model download

YouTube walkthrough (three real problems, code streaming live, tests going green):
https://www.youtube.com/watch?v=muq7VdgxqRk

Why this number matters

The Qwen team didn't publish HumanEval scores for any Qwen3-Coder variant — they consider the benchmark saturated and went straight to agentic ones (SWE-bench Verified, BFCL, Aider-Polyglot). For the 30B variant — the one that actually fits on a laptop — there were no published HumanEval/MBPP numbers. Until this run.

I also ran MBPP (sanitized): 83.3% pass@1 on a 168-problem sample. Pass rate stable since n=120; full 427-run was impractical because a few outlier tasks induce very long model responses (10+ minutes each).

Methodology

Setting	Value
Benchmark	HumanEval — 164 Python tasks (full)
Metric	pass@1 (first attempt only)
Temperature	0.0 — deterministic
Sampling	single sample per problem, no best-of-N
Execution	Python subprocess, 10s timeout
Hardware	M5 Max MacBook Pro · 128 GB unified memory
Model	Qwen3-Coder-30B-A3B-Instruct-MLX-8bit
Network	Wi-Fi OFF the entire run
Wall clock	14 minutes

For context — Qwen3-Coder 480B's official agentic benchmarks

The Qwen team's published numbers for the 480B flagship sibling (the bigger sibling of the 30B running on this MacBook):

Benchmark	Qwen3-Coder 480B	Claude Sonnet 4	GPT-4.1
SWE-bench Verified (500-turn)	69.6	70.4	—
Terminal-Bench	37.5	35.5	25.3
BFCL-v3	68.7	73.3	62.9
Aider-Polyglot	61.8	56.4	52.4

Source: Qwen team's official blog.

Why the offline part matters

If a tool needs the internet, three things are true:

Someone else can read what you sent.
Someone else can charge you for it.
Someone else can take it away.

If the same tool runs locally, none of those are true. That's a different category of software — and for law firms, medical practices, and accountants handling client material, it's the only legal one.

Reproduce it yourself

Open-source launchers: github.com/nicedreamzapp/claude-code-local
HumanEval dataset: github.com/openai/human-eval
Hardware: any M-series MacBook with ≥32 GB RAM (128 GB Max preferred for full 8-bit weights)
Total monthly cost: $0 after the model download

For law firms, medical practices, and accountants who want help getting this stack running on their own hardware — that's what AirGap is. 14-day pilot, fixed scope, the data never leaves your machines.

— matt

Originally published at Marijuana Union. For premium vaporizers visit iNeedHemp, wholesale at Nice Dreamz, and seeds at Tribe Seed Bank. Explore the 3D cannabis marketplace at The Farmstand.

Free AI on a MacBook vs $100-a-Month Claude Code — Hexagon Shootout

Matt Macosko — Thu, 23 Apr 2026 04:32:47 +0000

▶ Watch the race on YouTube: https://www.youtube.com/watch?v=2KeTDDodE0A

April 22, 2026. Anthropic's Claude Code Max plan jumped to $100 a month. I ran a live three-way AI race on the exact same prompt — Gemma 31B local, Llama 70B local, and Claude cloud — on a single MacBook, to see how close a free local stack gets to the paid cloud. Two of three contestants finished with zero cloud calls.

If you just want the video, it's here: FREE AI on a MacBook vs Claude Cloud — Hexagon Shootout.

If you want the repo, it's here: github.com/nicedreamzapp/claude-code-local.

Keep reading for the setup, the numbers, and the three things that surprised me.

The setup — same prompt, three contestants

Hardware: M5 Max MacBook Pro, 128 GB unified memory, Apple Silicon.

Gemma 31B — local, Apple MLX, 4-bit quantized (Google's code-specialized model)
Llama 70B — local, Apple MLX, 8-bit quantized (Meta's generalist)
Claude cloud — the real Anthropic API, using Claude Code unchanged

Same prompt to every contestant:

Build a single HTML file with inline JavaScript that shows a ball bouncing inside a rotating hexagon. Include gravity and realistic bounce physics.

Simple enough that the answer should be a few kilobytes of code. Interesting enough that it exposes how well a model handles real math — collision detection against rotating geometry, energy conservation, boundary clamping. When models trip, they trip here.

Every run was recorded end-to-end with a live stats panel: elapsed seconds, output bytes, tokens-per-second. No cherry-picking, no post-hoc edits to the physics code, no "here's what it SHOULD have said." What you see is what came out.

The results

Contestant	Time to ship working HTML	Tokens/sec	Cloud calls
Claude cloud	22 s	N/A (data center)	yes (via API)
Gemma 31B local	56 s	~30	zero
Llama 70B local	2:17	~11	zero

Claude cloud finished first — it's a data center somewhere. Gemma 31B finished clean in under a minute with working physics. Llama 70B took the longest and produced the most verbose output, but also landed a working demo in the end.

The headline isn't that one is "best." It's that two of the three ran with Wi-Fi that could have been off the entire time. That's the number that matters for anyone dealing with NDAs, PHI, client files, or just a flight without connectivity.

Three things that surprised me

1. Bigger isn't better when "bigger" is a generalist

I went in expecting Llama 70B to beat Gemma 31B on code quality. It's more than twice the parameter count. Gemma beat Llama cleaner and faster on this specific task.

Why: Gemma 4 is a Google model fine-tuned heavily for coding and math. Llama 3.3 70B is Meta's generalist — it's excellent at conversation, reasoning, creative writing, but it wasn't tuned to punch above its weight on HTML canvas physics.

If you're buying a local model for coding, you're better off with a 30B that's code-tuned than a 70B that's general. Don't count parameters, read the model card.

2. Claude Code's harness chokes local models

Claude Code (the CLI agent) sends a 29,000-token system prompt with 60 tool schemas in every request. That's tuned for the cloud — where a frontier model can happily chew through 30K tokens of context before even starting. On a local 70B, that prefill takes a minute or two before generation begins.

When I bypassed Claude Code and hit the MLX server directly with just the prompt, Llama 70B's wall-clock time dropped from 7+ minutes to under 2.

The tradeoff: without Claude Code's harness you lose the Write/Edit/Bash tool-use loop, so you can't use Claude Code as an agent, only as a generator. For research, benchmarking, or any single-shot prompt, direct is way faster. For actual coding sessions, the overhead is real but it's what buys you the agent loop.

3. Circle-approximation collision is the cheat code

All three models eventually produced a bouncing ball. The ones that worked used circle-approximation collision — treat the hexagon as a circle of its apothem radius for collision purposes, reflect velocity when the ball exceeds that radius, clamp the ball back to exactly inside. Five lines of math, reliable, hexagon can rotate as wildly as you want.

The ones that failed tried to do proper polygon-edge collision — compute the six edges of the rotating hexagon each frame, compute point-to-line distance for each, reflect off the appropriate edge. That's the "right" way, and it fails constantly because floating-point error lets the ball slip through edges during the rotation, and then the model doesn't know how to clamp it back.

I wouldn't have predicted this. The "simple" approximation is strictly better for the demo because it can't leak. For anything more complex than one ball, the polygon approach is necessary — but for a benchmark, approximation wins.

Who should care

Developers on laptops with 64+ GB of Apple Silicon unified memory: you can run this today, your hardware already supports it.
Anyone dealing with confidential work — lawyers, accountants, doctors, contractors handling NDAs or PHI: the cost isn't $0 vs $100, it's "does your data leave the machine" vs "does it not."
Frequent flyers and people who travel to places with bad internet: a 70B model on a laptop keeps working when the plane's Wi-Fi is $18 and throttled.
Anyone curious whether Apple's bet on unified memory was actually about AI: it was.

How to run it yourself

The repo is MIT licensed and open source. Full setup is in the README:

→ github.com/nicedreamzapp/claude-code-local

The project pairs a native-MLX Anthropic-API-compatible server with Claude Code. Point Claude Code at localhost:4000 and the official CLI talks to your local model as if it were the cloud API. Swap models with one env var. Ship code without the subscription.

Around 2,000 stars in the first month. If it's useful, a star helps.

TL;DR

Claude cloud: $100/mo, 22 seconds to a working hexagon.
Gemma 31B on my MacBook: $0, 56 seconds to a working hexagon.
Llama 70B on my MacBook: $0, 2:17 to a working hexagon.
Two of three ran with zero cloud calls.
Free AI on Apple Silicon is real, now, for a huge slice of what people use cloud APIs for.

The receipts, in video form: youtube.com/watch?v=2KeTDDodE0A

Originally published at Marijuana Union. For premium vaporizers visit iNeedHemp, wholesale at Nice Dreamz, and seeds at Tribe Seed Bank. Explore the 3D cannabis marketplace at The Farmstand.

"It Comes Out Of The Gate Very Fast": Disclosure Day Is An Action Movie

Matt Macosko — Mon, 20 Apr 2026 08:03:25 +0000

The moment Universal released the December 2025 teaser — wide Kansas sky, a meteorologist tilting her head, one note of John Williams score — the internet settled on an idea of what Disclosure Day was going to be. Slow. Sparse. Grown-up Spielberg. The Close Encounters of 2026. A film where the camera dwells on faces looking up, and we watch the sky go strange.

That idea was half right. Per Empire's exclusive for the June 2026 issue, Spielberg has other plans for the first 30 minutes.

"This movie comes out of the gate very fast. People who are expecting another slow-burn first act — this is not that movie."— Steven Spielberg to Empire

What We Know About the Opening

Based on the CinemaCon footage, the Super Bowl trailer, and the Empire cover package, the opening stretch of Disclosure Day includes:

A cold open in medias res. The first image, per reporters who saw the CinemaCon reel, is not a Kansas cornfield. It's a door being kicked in.

Josh O'Connor's fugitive run. Daniel Kellner already has the disclosure file when we meet him. He is not discovering anything in act one. He is running with it. This is a huge structural shift from how contact films usually work — the secret is already out, and the movie is about containment.

A car-chase-onto-a-train sequence. Confirmed by IMDb trivia and hinted at by O'Connor himself ("a car chase that is going to melt people"). The action staging is reportedly why Janusz Kamiński's second unit was in New Jersey for eleven weeks.

The Kansas City weather broadcast. The "click" sequence with Emily Blunt — previously assumed to be the film's quiet centerpiece — is actually in the first act. It's the inciting event, not the climax.

Why Spielberg Pivoted

David Koepp's prior Spielberg collaborations — Jurassic Park, War of the Worlds, Indiana Jones and the Crystal Skull — are all structured around the escalation of chase and threat. Koepp is not a meditative writer. He is a propulsion writer.

Spielberg telling Empire that the audience expectations have "caught up" to where the culture is means something specific: in 2026, the public already knows there are congressional UAP hearings happening. They already know Grusch testified. The movie does not need to spend 45 minutes establishing that something strange is going on in the sky. The audience is already there. So Spielberg is skipping that act and starting with the consequences.

The Close Encounters Comparison Breaks

If Close Encounters of the Third Kind spent half its runtime building to the Devils Tower meeting, Disclosure Day inverts it. The contact is the premise, not the ending. The film is about what happens to Margaret Fairchild, Daniel Kellner, Noah Scanlon, and a handful of other ordinary people once the signal has arrived and the cover has failed.

That is why Blunt's quote about "questions posed by Close Encounters" being "answered" works. Disclosure Day doesn't repeat the 1977 film's arc. It picks up where that film ended — and runs.

What It Means for the Box Office

Universal's tracking reportedly pushed for a more actioned-up back half of the marketing campaign after CinemaCon. Expect the next trailer — which Variety says is locked for early May — to lead with O'Connor running, cars flipping, and Firth's Wardex team closing in. The "look up at the sky" imagery isn't going away. It's just no longer the only mode. Disclosure Day is a summer action movie with a philosophical third act, not the other way around.

Sources

Empire — Disclosure Day Is An Action Movie That Comes Out Of The Gate Very Fast

Art Threat — Action-Packed Sci-Fi Thriller

Gizmodo — Mysterious Main Characters

Disclosure Day opens in theaters and IMAX on June 12, 2026.

Originally published at Disclosure Day Hub — the fan-built resource tracking Steven Spielberg's UFO film (June 12, 2026). Explore the full news hub, cast guide, and interview archive.

DEV Community: Matt Macosko

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

🧰The setup

🐛The bug

🔧The fix

🧠But did removing the big harness make it dumber?

📦Where it all lives

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

The setup

The bug

The fix

But did removing the big harness make it dumber?

Where it all lives

I wrote the missing Apple Silicon runtime for NVIDIA's Nemotron Omni

Verified, not asserted

Speed and memory

Why bother doing this locally

Try it

NVIDIA Shipped a Model That Sees and Hears — It Just Didn’t Run on a Mac. So I Wrote the Missing Piece.

What Nemotron Omni actually is

The part I keep having to relearn

Proving it, instead of vibing it

So does it actually work

Two things that fell out along the way

So who does this actually help?

Credit where it’s due

The Day Local AI Caught the Cloud: ds4, DeepSeek V4 Flash, and What Just Changed for Devs

The benchmark, for people who don’t want filler

Three architectural decisions in ds4 worth understanding

1. Asymmetric 2-bit quantization (only where quality is forgiving)

2. KV cache moved to disk (in 2026 SSDs are fast enough)

3. Pure Metal, not CUDA-with-a-shim

Why this matters for compliance-sensitive devs

How to actually run it

What this slots into

Where to follow

The IRS Said Yes — and What That Does and Doesn’t Mean

What it doesn’t mean

About the timeline, honestly

What it actually changes

Where this goes

M5 Max + 128GB = a 30B AI Coding Agent Running Locally. Wi-Fi Off.

Why the M5 Max changes the math

What you can do with it now

The full stack

The performance, honestly

Why the offline part matters

Benchmarks — actually run, not cited

For context — what the bigger sibling scores on harder benchmarks

Where this is heading

Try it yourself

4 AI Models Built the Same Game on One Laptop — and a Local One Beat the Cloud

The setup

What happened

The verdict

Run local models yourself (free)

Eight local AI agents on a Mac mini — and the product I'm building from them

The asset base

The gap that ate me for a month

The market gap, in numbers

The ladder

The three core agents

What's in the AirGap Box

Validation before scale

Honest year-1 range

What I will publish honestly

Links

Why I'm posting this

HumanEval on a MacBook — 81.7% pass@1, Wi-Fi off

Why this number matters

Methodology

For context — Qwen3-Coder 480B's official agentic benchmarks

Why the offline part matters

Reproduce it yourself

Free AI on a MacBook vs $100-a-Month Claude Code — Hexagon Shootout

The setup — same prompt, three contestants

The results

Three things that surprised me

1. Bigger isn't better when "bigger" is a generalist

2. Claude Code's harness chokes local models

Three architectural decisions in `ds4` worth understanding