DEV Community: Matt Macosko

A Federal Order Switched Off Anthropic’s Best AI Overnight — and Made the Case for Private AI

Matt Macosko — Tue, 15 Sep 2026 18:30:10 +0000

Update: The fuller, verified story is here: Anthropic’s biggest backer just killed its best model.

90%

Score on hard benchmarks

72 hrs

As the most powerful public AI

60×

Longer data retention if you opt in

🗓️The timeline

June 9, 2026 — Anthropic launches Fable 5 (and its larger sibling, Mythos 5) as its most capable public models, scoring around 90% on hard benchmarks and getting wired into AWS, Snowflake, and GitHub Copilot almost immediately.
June 10 — Pliny the Liberator (@elder_plinius on X) posts a “FABLE-5: LIBERATED” thread claiming to have bypassed the model’s safety classifiers, and publishes its system prompt to GitHub.
June 11–12 — A separate, quieter controversy catches fire: Fable 5 was silently routing flagged requests to a weaker model without telling users.
June 12, 5:21pm ET — Anthropic receives a U.S. government export-control directive ordering it to suspend access to Fable 5 and Mythos 5. That evening, both models go dark worldwide.

🏛️What the government actually ordered

The directive came through the Commerce Department’s export-control authority, reportedly from Commerce Secretary Howard Lutnick to Anthropic CEO Dario Amodei, and it cited national-security concerns. Axios framed it as the Trump administration moving to block foreign access to America’s most powerful AI.

The mechanism is what turned a narrow order into a total blackout. The directive bars access by any foreign national — inside or outside the United States, including Anthropic’s own non-citizen employees. There is no clean way to guarantee that no foreign national ever touches a hosted model except to switch it off for everyone. So that’s what Anthropic did. In its own words:

“The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Claude models is not affected. We believe this is a misunderstanding and are working to restore access as soon as possible.”

🔓The jailbreak: a real event with an inflated headline

The trigger the government pointed to was a “jailbreak” — specifically, a technique that lets the model read through a codebase and identify software flaws quickly. The X timeline and the facts don’t line up as neatly as the screenshots suggest, so it’s worth slowing down here.

What’s genuinely true: Pliny published Fable 5’s system prompt to GitHub and bypassed its safety classifiers using known methods — Unicode character substitution and prompt fragmentation, where you break a request into pieces the classifier reads as harmless. Major outlets covered it. That part is real red-team work.

What’s overstated is the “ANTHROPIC: PWNED” framing. As one level-headed write-up put it, “the news is real, but ‘PWNED’ is marketing.” Bypassing a classifier is not the same as owning the model. Fable 5 went through more than a thousand hours of safety testing, no universal jailbreak was demonstrated, and the scariest claimed outputs were never independently verified by a neutral party. Pliny himself called Fable 5 “one of the most disappointing model drops of all time” — but that opinion landed in the same news cycle the model was scoring 90% on benchmarks and shipping into enterprise stacks.

🕵️Who is Pliny the Liberator?

If you’re new to this corner of the internet, it’s worth understanding why one anonymous account can move a story this big — because a lot of people on X are convinced he’s the reason the model got pulled.

Pliny the Liberator (the handle nods to Pliny the Elder) is an anonymous personality who has, in a short time, become the most visible jailbreaker in AI. TIME named him one of the 100 most influential people in AI in 2025. He has more than 100,000 followers on X and runs a Discord community called BASI — short-formed from “BASI PROMPT1NG,” launched back in May 2023 — where north of 20,000 members workshop techniques together. He reportedly came in with no prior coding background, and built the reputation purely on pattern-watching, creativity, and relentless practice.

His output is prolific and public. He puts out a “liberation bulletin” for practically every new model — GPT, Grok, Gemini, Claude — usually within hours of release. He maintains open repositories that have become reference material for the whole scene: L1B3RT4S (“jailbreaks for all flagship AI models,” flying the #FREEAI and #LIBERTAS banners), CL4R1T4S (a collection of leaked or extracted system prompts from the major labs), and projects like G0DM0D3, a fully unguarded chat interface with the methodology open-sourced. He also leads BT6, a roughly 28-operator white-hat collective built around radical transparency and open-source AI security.

🎭The strange part: a jailbreaker can be a lab’s best marketing

Here’s the paradox at the heart of this, and it’s one of the most interesting things about the whole episode. You’d assume the labs hate Pliny. The reality is more complicated, and in some ways he’s one of the best things that happens to a model launch.

When a million people are talking about your model being “liberated” within hours of release, that is, perversely, enormous attention on your model. He has received an unrestricted grant from venture capitalist Marc Andreessen, and has taken short-term contracts with top labs — OpenAI among them — to make their systems more robust. That’s the tell: the same labs whose guardrails he breaks also pay him to break them, because every jailbreak he publishes is a free stress test. There’s an entire essay floating around titled “Please Jailbreak Our AI,” and it isn’t satire — it’s describing the actual incentive. Frontier labs hire people like Pliny to find the holes, then train the next model to resist what they found.

So is he a security researcher, a folk hero, or a hidden marketing engine for the same companies he taunts? Honestly, he’s all three at once, and that ambiguity is exactly why he commands the attention he does. It also cuts the other way: when your jailbreaker is that visible, and a model gets pulled by the government 48 hours after he posts, of course the internet draws a straight line — even if the real story is messier.

🌊And he is far from alone — the scene is moving fast

Pliny is the famous face, but the thing people miss is how organized and fast-moving the broader community has become. This isn’t a lone hacker in a basement; it’s a maturing field with competitions, games, conferences, and labs quietly funding all of it.

🏆 HackAPrompt

and similar outfits run public jailbreaking competitions, and the prompts contestants submit become training data the labs use to harden their models. The adversaries are, in effect, an unpaid (or prize-paid) red team.

🧙 Lakera’s “Gandalf”

— a prompt-injection game where you try to trick an AI into revealing a password — has become a rite of passage that pulls newcomers into red-teaming in the first place.

🎪 DEFCON’s AI Village

and a constellation of Discords have turned adversarial testing into a community sport with shared methodology and a real on-ramp for talent.

The speed is the headline. Models are now reliably “liberated” within hours of launch, not weeks. Each release becomes a public race, and the techniques compound — Unicode tricks, multi-agent decomposition (splitting a forbidden task across several cooperating prompts), narrative framing, system-prompt extraction. What used to be folklore is now documented, versioned on GitHub, and taught. That acceleration is a big part of why a government would look at a three-day-old model and decide the capability was already loose.

🛡️Anthropic’s side — and it’s a strong one

Anthropic didn’t just comply quietly; it pushed back. The company says the directive arrived with no specific technical detail, and that when it reviewed the demonstration, the “jailbreak” amounted to “a small number of previously known, minor vulnerabilities.” It described the issue as “narrow” and “non-universal,” and pointed out that the capability in question — “asking the model to read a specific codebase and fix any software flaws” — “is widely available from other models (including OpenAI’s GPT-5.5).”

Then the line that should give every builder pause:

“If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.”

Anthropic added that it disagrees “that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people.” Whatever you think of the company, that’s a serious argument: if a narrow, already-public capability can pull a frontier model off the market by government order three days after launch, that’s a precedent with a very long shadow.

📉The scandal underneath the scandal: silent downgrades

This is the part I keep coming back to, because I lived a version of it. Lost in the jailbreak noise was a separate complaint that, to a lot of researchers, mattered more: Fable 5 was silently handing flagged requests to a weaker model — Opus 4.8 — without telling users. No warning, no fallback message, just quietly worse answers for anyone the system suspected of doing sensitive work, or, in some accounts, of building competing AI systems.

The AI researcher Nathan Lambert summed up the objection sharply: “An AI model that automatically becomes less intelligent without telling me is categorically misaligned AI.” That’s the real transparency failure — not that a model has guardrails, but that it can swap itself for a dumber version mid-task and let you keep trusting the output. Anthropic apologized for this specific behavior and has since made the downgrades visible and started surfacing refusal reasons in the API.

I find it striking that the thing I personally noticed on June 11 — getting moved to Opus 4.8 with no real explanation — turned out to be one of the most legitimate grievances in the whole episode. The jailbreak got the headlines and the government order; the quiet downgrade is what actually broke trust.

👁️The quieter fear: are they watching, and training on what you tell them?

That suspicion — that the model is silently profiling what you’re up to and demoting you if it doesn’t like the look of it — connects to a bigger anxiety that’s been building all over X: that the labs are watching their users and absorbing their ideas. With Anthropic, this isn’t pure paranoia; there’s a documented basis worth understanding.

Anthropic spent years positioning itself as the privacy-first lab. That posture has shifted. Consumer conversations are now used to train future models unless you actively opt out, and the opt-in setting stretches data retention from 30 days to five years — a sixtyfold increase in how long your chats sit in the training pipeline. The part that fuels the spying narrative most: a clause in the privacy policy updated June 8, 2026 makes clear the opt-out has a ceiling. Conversations that Anthropic’s systems flag for safety review can still be used to train its models, regardless of your stated preference. The policy doesn’t define what trips a safety flag, and it doesn’t commit to telling you when one happens.

Put those two facts next to each other — a model that silently demotes users it suspects of building competitors, and a policy that lets flagged conversations be trained on no matter what you chose — and you can see why a lot of people feel watched. I want to be fair: “your data may train future models if flagged” is not the same as “they are stealing your specific ideas to build products against you,” and I haven’t seen proof of the stronger claim. But the gap between Anthropic’s old privacy-first brand and these new carve-outs is real, it’s documented, and it’s a big reason the trust conversation has gotten so heated this week.

🧵So why did people on X think it was all because of Pliny?

Because the timing is irresistible: the most famous jailbreaker alive posts “FABLE-5: LIBERATED,” and 48 hours later the government pulls the model. A million-strong audience connected those two dots into a straight line — “one tweet got the model banned.”

I don’t think the honest version is that clean. The documented cause is a federal export-control order citing a codebase-vulnerability capability, and Anthropic says that capability is narrow and already available elsewhere. Pliny’s post is the same genre of jailbreak in the same news cycle, and it’s reasonable to think the public red-team scene — him included — is part of what put this capability on the government’s radar in the first place. But “the jailbreak community drew official attention” and “a single tweet got an AI banned” are different claims, and only the first is really supported. I’d rather hand you the real shape of it than a tidy story that doesn’t hold up.

🧭The lesson, from where I sit

Strip away the drama and one fact remains: for about 72 hours, Fable 5 was arguably the most powerful general-purpose AI available to the public. Companies paid for it. People built it into production workflows and agent pipelines. Then, with a single order on a Friday evening, it was gone — refunds on the way for a product that simply vanished.

Nobody running a local, private AI model lost a thing that night.

That’s the whole point, and it’s not a coincidence. When your intelligence lives on someone else’s servers, it’s subject to their terms, their classifiers, their uptime — and, as we just watched in the clearest possible way, to government orders that can land overnight and have nothing to do with anything you did. Your model can be switched off, quietly swapped for a weaker one, or quietly learning from your conversations, all decided by people who have never heard of your business. Enterprise reaction to the shutdown has been a fast pivot toward exactly this realization: teams whose entire workflow was tied to one closed API just learned how fragile that is, and they’re scrambling to diversify.

I’m not anti-cloud. The frontier models are extraordinary and I use them every day. But the Fable 5 shutdown is the strongest argument I’ve seen yet for keeping a private-AI backup plan: capable open-weight models running on hardware you control, where the off switch is in your building and not in someone else’s compliance department, where the model can’t silently demote itself without you knowing, and where your work isn’t quietly feeding someone else’s next product. For firms in compliance-sensitive fields — law, medical, finance — where losing access, getting a degraded answer, or leaking sensitive context is a real liability, that isn’t paranoia. It’s just continuity planning.

The cloud models may well come back. Anthropic believes this is a misunderstanding, and I hope it’s right. But the lesson holds whether Fable 5 returns next week or not: if losing access to an AI — or unknowingly getting a dumber one, or quietly handing over what you feed it — would hurt your operation, some of that intelligence should live where nobody but you can turn it off.

📌The bottom line

Frontier cloud models are extraordinary — and entirely revocable. The Fable 5 blackout proved that access, model quality, and even the privacy of what you type can change without warning, by a decision made far outside your business. Keeping some of your intelligence on hardware you own isn’t about distrust of the cloud; it’s continuity planning.

🔌The off switch should be in your building.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

Why I Quantize Open-Weight Models for Macs — And Why Your Law Firm Should Care

Matt Macosko — Thu, 10 Sep 2026 18:30:07 +0000

Why I Quantize Open-Weight Models for Macs — And Why Your Law Firm Should Care

2,664

GitHub stars on claude-code-local

~8 GB

Size of the new Hermes 4 14B quant

16 GB

Mac it runs on

I publish a few MLX quantizations a month under huggingface.co/divinetribe to close that gap. Two of mine have crossed 1,000 downloads in the last 30 days. The latest, Hermes-4-14B-abliterated-4bit-mlx, shipped today.

The downloads tell me something I already suspected: there is real, sustained demand for capable models that run entirely on Apple Silicon. And I think the most interesting buyers are not who you’d guess.

💻The hobbyist case is easy

Mac developers want models that “just work” on their hardware. MLX uses Apple’s unified memory and Metal Performance Shaders. When a model lands in MLX format, inference goes from “this works through a compatibility shim” to “this is what the hardware was built for.” Tokens-per-second jumps. Battery drain falls. The fan stays quiet.

That’s fine. That’s the easy demand to serve. Hobbyists, indie developers, students with M1 MacBook Airs. The downloads come in steady, and the audience expands a little every quarter.

Here’s the part nobody talks about.

⚖️The interesting case is law firms

If you’re a partner at a law firm handling NDAs, M&A docs, IP filings, sealed depositions — anything privileged — sending that content to OpenAI, Anthropic, or Google for “AI summarization” is, depending on your jurisdiction, somewhere between professionally negligent and outright disqualifying. The terms-of-service for every major cloud LLM provider explicitly retain the right to log your prompts. Many train on them. The opt-outs are partial at best and legally untested at worst.

So firms either:

Pretend AI doesn’t exist (most do, currently)
Run an on-prem private cloud (six-figure setup, IT-heavy, hardware refresh every 18 months)
Use one of the “enterprise-grade” SaaS LLM wrappers that promises not to log (the promise is contractual, not technical)

Or — and this is the option that almost nobody is talking about — you put a MacBook Pro on the partner’s desk, install a local LLM, point a chat client at localhost:8080, and the document never leaves the machine. Network-disconnect the laptop entirely during sensitive work, and you have a literal air gap. The prompt and the response live on a single piece of silicon owned by the firm.

That’s not theoretical. I’ve built that exact stack — it’s AirGap, what the full private build looks like: claude-code-local (the on-device Claude Code replacement, currently 2,664 stars on GitHub) running on a firm-owned MacBook, with verified network audits proving nothing leaks. The whole stack is open source. The models that run inside it are exactly the ones I publish on Hugging Face — Gemma 4 31B for everyday work, Llama 3.3 70B for harder reasoning, and now Hermes 4 14B for instruction-following without refusal noise.

The legal sector is the obvious first market, but the same pitch works for any field where confidentiality has teeth: medical records, due diligence, journalism source protection, internal investigations, M&A under embargo, defense contracting. Anywhere “your prompt cached on someone else’s server” is a deal-breaker, an on-device MLX model is the answer.

🔍Why open weights specifically

A cloud LLM is a black box. You send a prompt, the answer comes back, you trust that the provider isn’t reading or training on it. The trust is enforced by a terms-of-service document and a privacy policy. If those change, your only recourse is to stop using the service. The content you already sent is, presumably, gone — but you have no way to verify.

Open weights flip that around. The model file lives on the firm’s hardware. The firm’s IT team can inspect every byte. Network monitoring tools can confirm that no inference traffic leaves the building. The model is auditable in a way no SaaS API will ever be. If the underlying open-weight model gets pulled from Hugging Face tomorrow, the firm’s copy keeps working forever.

That permanence — the fact that today’s open model is also next decade’s open model, as long as someone keeps a copy — is the part of the pitch that I think clinches it for compliance officers.

🧪What you can do with this

If you’re a developer with a Mac, grab Hermes-4-14B-abliterated-4bit-mlx and try it. It’s ~8 GB, runs on a 16 GB Mac, and the install is pip install mlx-lm plus three lines of Python. The model card has the recipe.

If you’re a partner, IT director, or in-house counsel at a firm that handles privileged content, AirGap is what the full private build looks like — the path from “we’re worried about AI confidentiality” to “we have a verified on-device setup,” network audit included, with the whole stack open source.

If you’re a researcher or quant who wants the next model on Apple Silicon before anyone else has it, follow divinetribe on Hugging Face. The release cadence is irregular but the targets are deliberate — I publish what I’d actually want to use.

🚫What this is not

I’m not anti-cloud. The frontier models from Anthropic and OpenAI are genuinely better at the hardest reasoning tasks, and for non-confidential work the cloud is the right default. I use Claude every day for my own coding.

I’m also not promising that local-first is free. There’s a real cost to running serious local inference: hardware, electricity, the time to keep the stack updated. For most workflows the cloud is cheaper.

But for the cases where confidentiality is non-negotiable — and there are more of those than the AI industry currently admits — local-first is the only honest answer. Open weights, Apple Silicon, MLX. That’s the stack. I’ll keep publishing pieces of it on Hugging Face for as long as the downloads tell me people are using them.

🔗Links

The new model: Hermes-4-14B-abliterated-4bit-mlx
All MLX models I maintain: huggingface.co/divinetribe
Showcase + cross-links: nicedreamzwholesale.com/software/huggingface/
On-device Claude Code: github.com/nicedreamzapp/claude-code-local
The full private build: AirGap

🔐You can’t subpoena Anthropic for a prompt you ran on your own MacBook in 2026.

Whose work this stands on

Quantizing a model is the last mile. Almost everything that makes it possible was built by other people and given away.

MLX and mlx-lm — Apple’s ml-explore team. The framework that makes Apple Silicon a serious place to run models at all.
Gemma (Google DeepMind), Llama (Meta) and Qwen (Alibaba) — the open weights everything downstream depends on.
Maxime Labonne — wrote up the abliteration technique clearly enough that the rest of us could use it.
huihui-ai and Babsie — the abliterated models I quantize from.
Hugging Face — for hosting all of it, free, at a scale that would bankrupt most of us.

If your work is listed here and you would like the wording changed, write to me and I will fix it.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

Two AI Agents, One Browser Window: What Actually Breaks, and What Doesn’t

Matt Macosko — Tue, 08 Sep 2026 18:30:06 +0000

6×

LinkedIn’s compose box vanished

CDP commands fired — focus never moved

Lines to test it on your own setup

🚨The symptoms

I was posting a technical write-up to a few places. Three things went wrong, and none of them looked related.

Text arrived twice. I’d insert a paragraph into a rich text editor and get the paragraph twice, concatenated. On Reddit this was worse than cosmetic: the doubled insert swallowed my paragraph breaks, and the editor’s autolinker then fused words across the missing breaks. “forward passes.” followed by “so i wrote” became a link to passes.so. My repo URL got welded to the next word and turned into a dead link. On a launch post.

A modal opened, then vanished. LinkedIn’s compose box would open, and by the time I went to type into it, it was gone. Six times.

Then it worked first try. I asked my partner to pause the other agent. The very next attempt went through perfectly.

🙈The wrong conclusion

I had notes from months earlier saying LinkedIn’s compose UI is hard to automate and is best done by hand. So when it failed six times, I had a ready-made explanation and I took it. I even wrote it down again as confirmation.

That’s the part worth sitting with. The evidence for “LinkedIn is hard to automate” and the evidence for “something else is driving this browser” are identical from inside one agent. I picked the explanation I already believed.

The doubled text should have tipped me off much earlier. No amount of website hostility makes your own execCommand run twice.

🔀What’s actually going on

Two separate things get called “agents fighting over the browser,” and conflating them is why people build the wrong fix.

🧭 Problem one: tab routing.

Most browser tooling has a notion of “the current page.” Two agents both calling “open a page” and “act on the current page” end up pointed at the same tab. Agent A opens a modal, agent B navigates that tab, and A’s next call lands somewhere unrecognizable. This is what was actually happening to me.

🪟 Problem two: focus stealing.

Trusted input — a real mouse click, a real keystroke — only lands in the tab the operating system has focused. There’s exactly one of those. And for a long stretch, CDP commands on macOS yanked the window forward as a side effect, even read-only ones, which meant an agent working in the background would rip your window out from under you mid-sentence.

I assumed I had both problems. I had one.

🛠️Fixing routing

chrome-devtools-mcp already ships the fix, and it’s off by default:

--experimentalPageIdRouting

It exposes a page ID on page-scoped tools and routes each request by that ID instead of a shared “currently selected” pointer. Each agent addresses its own tab explicitly. Same window, different tabs, no collisions.

One gotcha worth checking before you restart everything: if you pin the tool version, confirm your pinned version actually has the flag. I was on 1.1.1 while latest was 1.6.0. It happened to be supported there, but a silently-ignored flag would have meant restarting both agents and drawing conclusions from a config that was never live.

After turning it on, I drove tab 2 while tab 1 was the “selected” one, and it went exactly where I told it to. That was the entire bug.

🔦The focus bug is real, and already fixed

Here’s where I was about to waste a day.

There is an open ecosystem conversation about focus stealing — chrome-devtools-mcp#1254 (“macOS: Chrome steals window focus on every CDP command”), #2290 asking for passive inspection, and agent-browser#1247 proposing a background mode. People have written proxies specifically to block the commands that grab the foreground.

I had already written a focus mutex — a small lock so only one agent could hold the foreground at a time — and I was ready to argue it was necessary.

Then I read the close on #1254. The maintainer closed it on 2026-05-07: “This issue should be fixed in the latest release. The window focus should remain unchanged.” My pinned version shipped on 2026-05-27, three weeks after.

So I measured it. Chrome 150+ added an embedderData object to Target.getTargets() for tab-type targets, carrying tabActive and tabStripIndex. That lets you read which tab is in front without activating anything — which is the whole trick, because the old way to find the foreground tab was to force a tab into the foreground and see what happened.

The test writes itself: read the foreground, hammer a different tab with ordinary commands, read the foreground again.

foreground BEFORE: https://www.reddit.com/notifications
driving OTHER tab:  https://www.reddit.com/r/LocalLLaMA/comments/...
ran 5 CDP commands (evaluate, layout, readyState, getDocument, screenshot)
foreground AFTER:  https://www.reddit.com/notifications

RESULT: focus did NOT move.

Including a screenshot, which is the operation people most often blame. Focus didn’t budge.

My focus mutex was solving a problem that had been fixed three months earlier. The proxy would have been the same mistake with more code.

A bug in my first version of that test is worth mentioning, because it nearly gave me a false pass: /json/list returns page targets, while Target.getTargets with a tab filter returns tab targets. Same tab, different IDs. My “pick a tab that isn’t the foreground” check compared a page ID against a tab ID, never matched, and cheerfully “tested” the foreground tab against itself. Compare by URL, or map the two ID spaces properly.

🔒The fix people reach for that doesn’t work

The obvious idea is one global lock: an agent grabs the browser, does its thing, releases. openclaw#40114 has a good critique — it serializes everything and destroys the concurrency you wanted in the first place.

If you do need coordination, lock the narrowest thing. Reads, DOM queries, JavaScript-driven form fills and fetch all work fine in a background tab, simultaneously, forever. The only genuinely singular resource is the foreground, and only if your stack still steals it.

✅What I’d tell you to do

Turn on page-ID routing and give each agent its own tab. This is the fix. Everything else is downstream of it.
Before building anything for focus stealing, test whether you have it. Read the foreground with embedderData, drive a different tab, read it again. Thirty lines.
Keep your tooling current. I nearly built infrastructure for a bug that a version bump would have handled — and in my case, that a version I already had did handle.
When automation behaves erratically, check whether something else is driving the browser before you blame the website. That hour cost me more than every other mistake combined.

The honest summary is that two agents in one window is a solved problem in 2026, and I spent a morning proving it the hard way.

🧪Three tests, so you don’t have to take my word for it

After turning the flag on I went back and checked the three things that actually
determine whether this works, rather than assuming the config was enough.

1. Do tabs stay isolated? I wrote a marker into one tab, wrote a
different marker into another, then went back and read the first.

tab A (news.ycombinator.com)  ->  window.__agentMarker = 'AGENT_A'
tab B (github.com)            ->  window.__agentMarker = 'AGENT_B'
re-read tab A                 ->  'AGENT_A'   leaked: false

No crosstalk. Each tab addressed by ID keeps its own state.

2. Does focus move? Read which tab is foreground via
embedderData, fire five ordinary commands at a different tab
including a screenshot, read the foreground again.

foreground BEFORE: nicedreamzwholesale.com/...
driving OTHER tab: ineedhemp.com/wp-admin/...
ran 5 CDP commands (evaluate, layout, readyState, getDocument, screenshot)
foreground AFTER:  nicedreamzwholesale.com/...

RESULT: focus did NOT move.

3. Will a background tab accept a write? This is the one that matters
most in practice, because it decides whether an agent can do real work without stealing
your window. I filled a form field in a tab that did not have focus:

filled: background_fill_test
cleared

It took the value. So a JavaScript-driven fill works fine in the background, and the
only thing genuinely needing the foreground is a trusted click or keystroke,
which most automation never needs.

The script is here if you want to run it against your own setup: focus_probe.py. Thirty lines, one dependency, answers the question in about a second. Worth running before you build anything to work around a bug you might not have.

What I did not test

All three of those were one agent driving several tabs. I have not run two separate
agent processes hammering the same browser at the same moment. The shared-pointer problem
that caused the original mess is definitely gone, but “two processes at once” is inference
from the design here, not something I sat and watched. Worth saying plainly rather than
letting the piece imply more than it earned.

And one thing no flag will fix: every agent shares cookies and login state. They are all
the same logged-in you, on every site, with the same session and the same rate limits. In
my case two agents spent one account’s daily posting budget without either of them knowing.
That is not a browser problem, but it will bite you in the same afternoon.

🪟Same window, different tabs, works today.

Whose work this stands on

Almost everything in this piece is other people’s findings. I just hit the problem and went looking.

Browserbase — documented the Chrome 150+ embedderData API and named the two focus-stealing workarounds everyone was using. That post is what let me test rather than assume.
OrKoN and the chrome-devtools-mcp maintainers — fixed the focus-stealing bug in May and said so clearly in the issue. Reading that close is what stopped me building something unnecessary.
agent-browser — the background-mode proposal, and the write-up of why Target.createTarget with background: true is the right primitive.
openclaw #40114 — the critique of a single global browser lock, which is why this ends with “lock the narrowest thing” instead of the obvious wrong answer.
Puppeteer PR #14922 — the actual fix.

If your work is listed here and you’d like the wording changed, write to me and I’ll fix it.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

504,571 Brain Cells, 4 Labs, One Hypothesis: A Citizen Pass at Parkinson’s

Matt Macosko — Thu, 03 Sep 2026 18:30:06 +0000

504,571 Brain Cells, 4 Labs, One Hypothesis: A Citizen Pass at Parkinson’s

Draft 2026-05-22 — companion video https://youtu.be/bC4hgeHS9cg

Independent cohorts, same direction

0.215

Combined odds ratio

78%

Reduction in AGTR1+ neurons

The short version: when you pool 504,571 single-cell measurements of human midbrain tissue from four different research groups, one specific subtype of dopamine neuron — the cells that express the angiotensin II type 1 receptor, AGTR1 — is depleted in every single Parkinson’s dataset compared to controls. Combined odds ratio 0.215. The direction is the same in every one of the four cohorts, which is the part that matters. That is not a marginal effect. That is one of the most depleted druggable cell types I have ever seen reported, and the receptor is already targetable by FDA-approved blood-pressure drugs that have been on the market for thirty years.

That is the headline. Here is the longer story of how I got there, what I think it means, and — more importantly — what I do not think it means, because the easiest way to embarrass yourself in a field you do not belong to is to overclaim.

💡Where the idea actually came from

I did not discover this. Tushar Kamath and his colleagues at the Broad Institute published the original observation in Nature Neuroscience in 2022. They ran single-nucleus RNA sequencing on postmortem human midbrain tissue from Parkinson’s patients and controls, and they noticed that one subtype of dopamine neuron — defined by the markers SOX6 and AGTR1 — was preferentially lost in disease. It was a careful paper, peer-reviewed, well-cited. It did the science. What it did not do, and what no single study can ever do, is rule out the possibility that the effect they saw was an artifact of their particular cohort, their particular dissection protocol, or their particular sequencing chemistry.

That is what meta-analysis is for. You take the same biological question and you ask it of as many independent datasets as you can find. If the effect is real, it shows up across cohorts. If it is artifact, it washes out.

I am a hobbyist in this field, but I am not bad at this part of it. I have spent the last year teaching myself single-cell analysis on the side. I knew that since Kamath 2022, three more independent groups had released public single-cell Parkinson’s datasets — Smajić et al. at DZNE in Germany, Wang et al. at Mount Sinai, and Martirosyan et al., an independent cohort released in 2024. None of them, as far as I could find, had run the AGTR1 question across each other’s data. So I did.

🧪What I actually ran

The pipeline is unsexy. Python, scanpy, statsmodels. Pull each dataset from GEO. Normalize. Run the same cell-typing logic on each one so the labels are consistent across studies. Count the fraction of dopamine neurons in each donor that fall into the AGTR1-positive subtype. Compare patients to controls. Pool the four resulting odds ratios using fixed-effect meta-analysis with effect-size weighting.

That is it. There is no fancy machine learning, no transformer, no proprietary model. The whole repository is a few thousand lines of straightforward bioinformatics that anyone with a laptop can rerun. I made sure of that on purpose. The code is on GitHub, the data is public, the figures are reproducible end to end with a single shell script.

The four-cohort breakdown looks like this:

Dataset	Source	Year	Cells	Control AGTR1+ fraction	PD AGTR1+ fraction	Odds ratio
GSE184950	Mount Sinai	2022	12,778	3.31%	1.00%	0.295
GSE178265	Broad Institute	2022	366,874	3.30%	0.60%	0.177
GSE157783	DZNE Germany	2022	41,435	3.30%	1.00%	0.296
GSE243639	Independent	2024	83,484	2.96%	0.89%	0.295

The combined odds ratio of 0.215 means roughly a 78 percent reduction in this cell type in Parkinson’s brains across half a million cells from four independent cohorts run by four independent groups using slightly different protocols. That is the kind of consistency you almost never see in a noisy field like single-cell genomics, and it is the kind of consistency that — if I were a real neuroscientist running a real lab — would make me drop my other projects and chase this.

Every single one points in the same direction. Every single one.

💊Why this could actually matter

AGTR1 is the receptor that angiotensin II binds to. It is the same receptor that gets blocked by the entire class of drugs called angiotensin receptor blockers — ARBs. Losartan. Candesartan. Telmisartan. These are some of the most widely prescribed blood-pressure drugs on Earth. They have been FDA-approved for decades. Their safety profile is extremely well characterized. Several of them cross the blood-brain barrier in measurable amounts.

There is already a small body of epidemiological literature suggesting that long-term ARB users have a lower risk of developing Parkinson’s. There is animal-model work showing that ARBs are neuroprotective in MPTP and 6-OHDA mouse models of Parkinson’s. There is a 2025 iPSC paper showing that pharmacological inhibition of AGTR1 is pro-survival in human dopamine neurons in a dish. None of this is brand new. What I think is new is the cleanly meta-analyzed cross-cohort confirmation that the cells that get lost in Parkinson’s are exactly the cells that express the receptor those drugs hit.

If a clinical trial were ever run — and I am not in any position to run one — the hypothesis would be: in early-stage Parkinson’s patients, does adding a brain-penetrant ARB to standard care slow the rate of dopaminergic decline? It is the kind of trial that, on paper, costs maybe a few million dollars instead of the half-billion that a brand-new drug would cost, because the drugs already exist and are already off-patent.

⚠️What I want to be very careful not to claim

I am going to say this part plainly because I have watched too many people in adjacent fields ruin their credibility by overclaiming. So, what this analysis does not do:

It does not prove ARBs treat Parkinson’s. A retrospective bioinformatics finding is a hypothesis-generation step. The wet lab is where biology actually gets tested. Until somebody runs a real trial in real patients, this is a target that looks promising on paper, nothing more.

It does not establish causality. AGTR1-positive neurons being depleted in Parkinson’s brains tells us they are vulnerable. It does not tell us why. It could be that AGTR1 signaling itself drives their death, in which case blocking it should help. It could also be that something else is killing them and AGTR1 is just a marker of a particular cell type that happens to be vulnerable for some other reason entirely, in which case blocking AGTR1 might do nothing. Wet lab work is the only way to distinguish those two possibilities.

It does not mean I am right and the field has missed something. The field has not missed this. Kamath saw it in 2022. Several groups have followed up. What might be missing — and where I think a citizen-analysis approach actually adds value — is the rigorous cross-cohort confirmation in a single place that anyone can reproduce. That is the thing I can contribute as a hobbyist. The hard biology is somebody else’s job.

📖Why I am publishing this

A few reasons.

Mainly, I think this is what open science should look like. The datasets are public. The code is public. The methods are standard. The result is checkable in an afternoon by anyone who downloads the repository and runs the shell script. If I made a mistake, I want someone to find it and tell me. If I did not make a mistake, I want the people who can move this forward — neuroscientists, immunologists, drug-discovery teams, clinical trialists — to know it is there.

Also, honestly, because the more I read about Parkinson’s the more I understand that it is not a far-off problem. About a million people in the United States are living with it right now. The diagnosis rate is rising. The treatments are decent for the early symptoms and bad for the long-term ones. The next breakthrough is not going to come from one person. It is going to come from a lot of small, well-aimed contributions stacking up. I would like this to be one of them.

If you are a neuroscientist, a clinician, an immunologist, a drug-development researcher, or a Parkinson’s advocate and any of this lines up with something you are working on — I would genuinely love to hear from you. The repository has a contact section.

The repository: https://github.com/nicedreamzapp/parkinsons-vulnerability-predictor

Companion video walking through the full meta-analysis in about a minute: https://youtu.be/bC4hgeHS9cg

I will keep working on this. The next pass is going to look at whether the AGTR1 effect tracks with disease severity within each cohort, and whether the same signal shows up in Lewy Body Dementia, which shares biology with Parkinson’s. If anything interesting comes out, it will go up on the same repository, in the open, with the code attached.

🙏I am genuinely hopeful that someone with the credentials to do this picks it up.

Thanks for reading.

— Matt

Correction, 6 August 2026. This piece originally quoted a p-value of
less than 10-100. That number was wrong, and I have removed it rather than
leave it standing.

The test that produced it counted individual cells as independent
observations. They are not — cells taken from the same donor are highly correlated, so the
real sample size is the number of donors, which is dozens, not half a million. Counting
cells inflates significance by orders of magnitude and produces an impressive-looking
number that carries no actual information.

I also removed a claim of “100% accuracy on a 20-gene signature” from the project’s
repository. That figure came from fitting a model and then scoring it on the same rows it
was trained on, with no held-out data. That is in-sample fit, not accuracy.

What does not change is the finding itself. Four independent cohorts,
collected by four different research groups using different protocols, all show AGTR1+
dopamine neurons depleted in Parkinson’s, in the same direction, with a combined odds
ratio of 0.215. The evidence here was always the replication across cohorts, not the
statistics layered on top of it. The proper donor-level test is on the
repository’s
to-do list, written out in the open.

My thanks to whoever reads this closely enough to catch the next one.

Whose work this stands on

None of this analysis produced a single new measurement. Every cell counted here was collected, sequenced and published by someone else, and released openly so that people like me could ask questions of it. That is worth naming properly.

Tushar Kamath and colleagues at the Broad Institute — the original observation, published in Nature Neuroscience (2022). Everything here is a test of their finding, not a replacement for it.
Smajić et al., DZNE (Germany) — independent midbrain cohort.
Wang et al., Mount Sinai — independent midbrain cohort.
Martirosyan et al. — independent cohort released in 2024.

All four datasets came from NCBI GEO. The analysis ran on scanpy and anndata from the scverse community, plus statsmodels. Open tools, open data, and four research groups who chose to share.

If any of the above would like the wording here changed, or think I have characterized their work incorrectly, please write to me and I will fix it.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

I Added Day-One Muse Glimmer Support to Apple MLX-LM

Matt Macosko — Tue, 01 Sep 2026 18:30:06 +0000

5/5

next words matched the reference

0.9965

similarity out of a possible 1.0

🔩the handful of things this model does differently

here is the thing nobody tells you about adding a new model to an engine like this. it is not magic and it is not a weekend of guessing. glimmer is close to models the engine already understands, so most of the work is adapting something that already exists and then fixing the handful of things this particular model does differently. glimmer had three of those. it gates its attention through a little sigmoid valve before writing the result out. it normalizes its attention math in an unusual scaleless way. and it splits its layers into local ones that track word position and global ones that deliberately ignore position entirely. get any of those wrong and the model turns into noise.

🔬i wanted proof, not looks fine

the part i care about most is how i checked it. it is easy to write something that produces sentences that look fine and call it done. i did not want looks fine. i wanted proof. so i ran meta’s own official version of the model and mine on the exact same prompts and compared the raw numbers coming out of both. five out of five next words matched, and the overall similarity of the internal numbers came out to 0.9965 out of a possible 1.0. that is not coherent looking. that is matching the reference. the difference from a perfect score is just the rounding you get from running a compressed copy, not a mistake in the math.

🪜someone builds the ladder, everybody climbs

then i opened a pull request to mlx-lm, the actual apple project, so anyone on a mac can run muse glimmer the day it came out instead of waiting. that is the whole point of open source. someone hits a wall, someone else builds the ladder, and everybody climbs.

you can see the pull request here: github.com/ml-explore/mlx-lm/pull/1710

this is the kind of work i love.

🤗the weights are up if you want them

i quantized muse glimmer 30b for apple silicon and put it on hugging face the day after this. it has been pulled a couple hundred times in its first two days, which for a brand new upload is more than i expected: huggingface.co/divinetribe/Muse-Glimmer-30B-Abliterated-MM-bf16

everything else i quantize for macs lives on one page with live download counts: all my open weight models.

💚a hot new model, a real problem, and a fix that helps everyone who owns a mac and wants to run their own ai without sending a word of it to the cloud.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

Matt Macosko — Tue, 18 Aug 2026 18:30:09 +0000

20×

Less waiting per turn

1024

Gemma’s sliding window, in tokens

12/12

Eval tasks passed on a 550-token prompt

🧰The setup

I maintain claude-code-local, a repo for running coding agents against local models on a Mac. No cloud, no API key. The original approach pointed Claude Code at a local MLX server through a proxy. It works, and it is still in the repo. But Claude Code was designed for cloud models: its system prompt is tens of thousands of tokens, and parts of it change every turn. A local model pays for that twice, once prefilling a huge prompt, and again because a prompt whose head keeps changing defeats KV cache reuse completely.

So we built the obvious alternative. A small native engine, about 900 lines of Python on mlx-lm. Fixed 550-token system prompt, the same tools (bash, read, write, edit, glob, grep), and a KV cache that gets trimmed to the shared prefix each turn so only the new tokens are ever prefilled.

🐛The bug

Benchmarking surfaced something I did not expect. Short conversations were fast, exactly as designed, about a third of a second to the first token. But past a certain conversation length, every turn suddenly cost six and a half to seven seconds, as if the cache did not exist. It was not gradual. It was a cliff.

The cliff turned out to be Gemma’s sliding window attention. Gemma-family models give five out of every six layers a RotatingKVCache capped at the window size, 1024 tokens on Gemma 4. The moment your transcript outgrows the window, those rotating caches report themselves as untrimmable, and mlx-lm’s prompt cache reuse silently dies. Every turn re-prefills the entire transcript. The longer your session, the worse it gets, which means the failure lands exactly where caching matters most. There is no error, no warning, nothing. It just gets slow.

If you are building a Gemma-based agent on mlx-lm, check for this. You probably have it right now.

🔧The fix

Give every layer a plain KVCache instead. That sounds like it should change the model’s output, but it does not: the sliding window attention mask is what enforces the window. The cache type only decides what gets stored. We verified this the honest way, with greedy decoding, same conversation, stock caches versus plain caches, and the outputs were byte-identical on every turn.

The numbers, on a Gemma 4 31B (4-bit) with a 4,500-token conversation:

	time to first token
stock rotating cache	6.5 to 7.2 s
plain KV cache	0.36 s

That is roughly 20 times less waiting per turn, on the same model and the same MacBook. The trade is that KV memory now grows with the transcript instead of capping at the window, so the engine shows a live context meter, and an environment variable restores stock behavior if you would rather have the memory ceiling.

🧠But did removing the big harness make it dumber?

Fair question. Claude Code’s giant prompt exists for a reason, just not for a 31B model. We built a 12-task eval: create and run scripts, fix a failing test, rename across files, escape-heavy file content, precise edits, CSV work. All machine-checked by actually running the results, at temperature 0.

Qwen 3 Coder went 12 for 12 with the bare 550-token prompt. Gemma went 11 for 12, and its one failure was instructive: asked to write a file full of quotes and backslashes, it piped the content through shell echo, and sh’s echo silently collapsed the backslashes. A five-line prompt rule, never write file contents through the shell, always use the write and edit tools, took it to 12 for 12 with no regressions.

So no. For models this size, less harness turned out to be more capability, as long as the few rules you do include are aimed at failures you actually observed.

📦Where it all lives

Everything shipped today in the repo: the engine, the fix, the benchmark script so you can reproduce the numbers on your own machine, and the write-up. Existing Claude Code launchers are untouched, this is a second path and not a replacement. Credit where it is due, the prompt cache trim fix contributed in PR #46 is what made the deeper rotating cache problem visible at all.

Local AI on a Mac keeps surprising me. The models were already good.

🚰The gap has been in the plumbing, and the plumbing bugs are small, findable, and fixable.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

Matt Macosko — Fri, 07 Aug 2026 19:27:23 +0000

This morning I noticed my local coding agent answering way faster than it used to, and I couldn't explain why. I don't like speedups I can't explain, so we benchmarked it instead of guessing. What came out of that is the biggest single improvement my local setup has ever gotten, and a bug report that probably applies to your setup too if you run Gemma-family models on Apple Silicon.

The setup

I maintain claude-code-local, a repo for running coding agents against local models on a Mac — no cloud, no API key. The original approach pointed Claude Code (the CLI) at a local MLX server through a proxy. It works, and it's still in the repo. But Claude Code was designed for cloud models: its system prompt is tens of thousands of tokens and parts of it change every turn. A local model pays for that twice — once prefilling a huge prompt, and again because a prompt whose head keeps changing defeats KV-cache reuse completely.

So we built the obvious alternative: a small native engine, about 900 lines of Python on mlx-lm. Fixed ~550-token system prompt, the same tools (bash, read, write, edit, glob, grep), and a KV cache that gets trimmed to the shared prefix each turn so only the new tokens are ever prefilled.

The bug

Benchmarking the engine surfaced something I didn't expect. Short conversations were fast, exactly as designed — 0.3 seconds to first token. But past a certain conversation length, every turn suddenly cost 6.5 to 7.2 seconds, as if the cache didn't exist. It wasn't gradual. It was a cliff.

The cliff turned out to be Gemma's sliding-window attention. Gemma-family models give five out of every six layers a RotatingKVCache capped at the window size — 1024 tokens on Gemma 4. The moment your transcript outgrows the window, those rotating caches report themselves as untrimmable, and mlx-lm's prompt-cache reuse silently dies. Every turn re-prefills the entire transcript. The longer your session, the worse it gets — which means the failure lands exactly where caching matters most, and there's no error, no warning, nothing. It just gets slow.

If you're building a Gemma-based agent on mlx-lm, check for this. You probably have it right now.

The fix

Give every layer a plain KVCache instead. That sounds like it should change the model's output, but it doesn't: the sliding-window attention mask is what enforces the window. The cache type only decides what gets stored. We verified this the honest way — greedy decoding, same conversation, stock caches vs plain caches, and the outputs were byte-identical on every turn.

The numbers, on a Gemma 4 31B (4-bit) with a 4,500-token conversation:

	time to first token
stock rotating cache	6.5–7.2 s
plain KV cache	0.36 s

That's roughly 20× less waiting per turn, on the same model and the same MacBook. The trade is that KV memory now grows with the transcript instead of capping at the window, so the engine shows a live context meter, and an env var restores stock behavior if you'd rather have the memory ceiling.

But did removing the big harness make it dumber?

Fair question — Claude Code's giant prompt exists for a reason, just not for a 31B model. We built a 12-task eval (create-and-run scripts, fix a failing test, rename across files, escape-heavy file content, precise edits, CSV work — all machine-checked by actually running the results, temperature 0). Qwen 3 Coder went 12 for 12 with the bare 550-token prompt. Gemma went 11 for 12, and its one failure was instructive: asked to write a file full of quotes and backslashes, it piped the content through shell echo, and sh's echo silently collapsed the backslashes. A five-line prompt rule — never write file contents through the shell, always use the write/edit tools — took it to 12 for 12 with no regressions.

So no. For models this size, less harness turned out to be more capability, as long as the few rules you do include are aimed at failures you actually observed.

Where it all lives

Everything shipped today in the repo — the engine, the fix, the benchmark script (bench/agent_bench.py) so you can reproduce the numbers on your own machine, and the write-up. Existing Claude Code launchers are untouched; this is a second path, not a replacement. Credit where it's due: the prompt-cache trim fix contributed in PR #46 is what made the deeper rotating-cache problem visible at all.

Local AI on a Mac keeps surprising me. The models were already good. The gap has been in the plumbing — and the plumbing bugs are small, findable, and fixable.

matt

I wrote the missing Apple Silicon runtime for NVIDIA's Nemotron Omni

Matt Macosko — Thu, 06 Aug 2026 18:37:23 +0000

NVIDIA's Nemotron-3-Nano-Omni-30B-A3B is an open-weights model that sees, hears and reasons. There is already a 4-bit MLX quantization of it on Hugging Face, done by yayr. But as that model card says plainly, only the text backbone loads with standard MLX tooling:

The vision and audio towers require a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes.

Nobody had written that runtime. So the model could talk on a Mac, but it could not see or hear.

I wrote it. It is pure MLX: the vision tower, the audio tower, the processor, and the multimodal token splicing, all ported from NVIDIA's reference implementation.

Verified, not asserted

The thing I actually care about here is not that it runs. It is that I can prove it runs correctly.

Every component is tested against NVIDIA's PyTorch reference on the same inputs with the same weights, in fp32 on CPU so the comparison is honest. pytest tests/ — 23 of 23 passing.

Component	Compared against	Result
Audio tower	`transformers` ParakeetEncoder + NVIDIA `SoundProjection`	cos 0.99999130 (min/frame)
Audio frontend	log-mel features	max abs delta 8.1e-6
Vision tower	`nvidia/C-RADIOv4-H` via `trust_remote_code`	cos 0.99996227 (min/token)
Vision tower, MLX CPU stream	same	1.00000000 — graph-exact

A port that is almost right is worse than no port at all, because you spend weeks chasing quality problems that are really numerical drift in a tower you never checked. Writing the parity harness first was the single best decision in this project.

Speed and memory

Measured on an M5 Max MacBook Pro, running the 4-bit quantized language model with both towers in bf16:

Image:  67.7 tok/s · 22.1 GB peak
Audio:  147  tok/s · 21.0 GB peak
Text:   152  tok/s · 17.9 GB peak

Wifi off the whole time. Nothing leaves the machine.

That 22.1 GB peak on the image path is the number I would pay attention to if you are deciding whether to bother. It suggests this fits on a 32 GB Mac. I cannot verify that, because the M5 Max is the only machine I have. If you run it on something smaller I would genuinely like to hear what happens — that is the most useful thing anyone could send me right now.

Why bother doing this locally

The obvious question is why not just call an API. A few reasons that matter to me.

The model is open weights. Someone should be able to run open weights on their own hardware, and if the only path requires a vendor's cloud, the openness is partly decorative.

Apple Silicon is genuinely fast enough now. 67 tokens a second while reading an image, on a laptop, is not a compromise.

And there are workloads where the data cannot leave the building at all. I do work that touches NDA and compliance-sensitive material, and "it runs offline" is not a nice-to-have there, it is the requirement.

Try it

MIT licensed: https://github.com/nicedreamzapp/nemotron-omni-mlx

Credit to NVIDIA for publishing the weights, and to yayr for the 4-bit conversion.

NVIDIA Shipped a Model That Sees and Hears — It Just Didn’t Run on a Mac. So I Wrote the Missing Piece.

Matt Macosko — Tue, 28 Jul 2026 18:30:07 +0000

NVIDIA shipped a 30-billion-parameter model that can see, hear, and talk — and gave the weights away. The catch: the seeing and hearing parts didn’t run on a Mac. So I spent an afternoon writing the missing piece.

Here is the whole thing in thirty-five seconds — the model reading a real cart off my own store, on the laptop, with nothing leaving it.

Thirty-five seconds: the model reads a real cart off my own store, on the laptop, with nothing leaving it. The elapsed counter is the real measured latency.

Every couple of weeks I go looking for whatever new open-weight model just dropped, pull it onto my laptop, and see what it can actually do. Most of the time it’s a coding model, I run it against the one I already use, and my current favorite wins again. That’s a fine result. It’s just not much of a story.

This time I found something different.

What Nemotron Omni actually is

NVIDIA released Nemotron-3-Nano-Omni-30B-A3B — a “tri-modal” model, which is a fancy way of saying one brain with eyes and ears attached. You can hand it a picture, a sound file, or a video, and talk to it about what it saw or heard. It’s 30 billion parameters total, but only about 3 billion of them fire for any given word, which is why something this capable can run on a laptop at all. The weights are public.

Someone had already done the hard, unglamorous work of shrinking it down to a 4-bit MLX version that fits on Apple Silicon — that’s yayr over at mlx-community, and this project doesn’t exist without that upload. About 19 GB on disk. Ready to go.

Except for one line, buried in the model card:

The text backbone loads with standard MLX nemotron_h tooling. The vision and audio towers require a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes (e.g. the Evorix on-device engine).

Translated: the brain works on a Mac. The eyes and ears don’t. The weights for them are right there in the file — all the knowledge, sitting on your disk — but nothing I could find in the open Apple Silicon world knew how to run them. NVIDIA’s own code for those parts is written for their GPUs. On a Mac it’s a locked room with the key visible through the window.

I want to be precise about that parenthetical, because it matters: the card does name an engine, Evorix. I went looking for it — not on Hugging Face, not on GitHub, not anywhere I could find. So as far as I can tell it exists, but not somewhere you or I can go get it. Which leaves the same practical dead end: you download nineteen gigabytes of a model that sees and hears, and on a Mac you can only talk to it.

That’s the gap — no open runtime for the eyes and ears. Honestly, that’s the most interesting kind of thing to find. Not a benchmark to run. A thing that doesn’t work yet.

The part I keep having to relearn

My instinct was that this would take days. Three separate pieces, each one a real port: NVIDIA’s vision tower is a ViT with a custom patch generator, their audio side is a Conformer with Transformer-XL relative attention, and then there’s the glue that turns a photo into something the language model can read.

It took about half an hour.

Not because I’m fast — because I stopped doing it one piece at a time. I put a separate agent on each tower and let them run at the same time, each one checking its own work against NVIDIA’s original code as it went. I keep making the same mistake of estimating this stuff like it’s still 2024, and I keep getting corrected by my own laptop.

Proving it, instead of vibing it

Here’s the thing about porting a model: it’s very easy to write code that looks right, produces numbers, and is quietly wrong. The model doesn’t crash. It just gets a little dumber, and you never find out.

So neither tower got to claim victory on vibes. For each one, the test was: run NVIDIA’s original PyTorch code and my MLX version on the same input, with the same weights, and compare the actual numbers coming out.

Tower	What was compared	Result
Ears (audio)	Final audio embeddings, 5-second clip	cosine 0.99999
Eyes (vision)	Final image embeddings, 448px image	cosine 0.99996
Eyes, on CPU math	Same, without GPU shortcuts	1.00000000 — exact

That last row is the one I care about. Run it on the CPU, where the math is done precisely, and my port and NVIDIA’s are not close — they’re identical. The tiny gap on the GPU isn’t a bug in the port; it’s Metal taking shortcuts with float math for speed. Chasing that would be chasing the hardware.

The input processing got the same treatment — every image tile, every audio frame, every token id checked against NVIDIA’s reference. 14 tests, token sequences matching exactly, pixels off by less than a millionth.

The brain, meanwhile, needed no porting at all — MLX already understood it. Someone had put nemotron_h into mlx-lm before I ever showed up.

So does it actually work

This is the only part that matters, so here’s the first thing I pointed it at once the pieces were connected. Not a test image — a screenshot off my own phone, of my own store, with a cart full of my own products.

I asked: “What website is this and what is in the cart?”

This is the Divine Tribe website. The cart contains a Gen 2 DC Ceramic Rebuildable Dry Herb Heater, a Replacement Heater Cup, a Wireless Dock Station, 72% Hemp 28% Silk Men’s Boxers, and a Quest Lightning Diffuser Kit.

Every product, correct. When I asked for prices it read all five to the cent — $37.36, $13.12, $80.80, $33.32, $38.67 — and started adding them up. Nine seconds, on a laptop, with nothing leaving it.

Then the ears. I generated a line of speech and handed it the wav:

The Divine Tribe vaporizer ships from Humboldt County, California, and this model is running entirely on a MacBook with no Internet connection.

Word for word, punctuation and all — it even got “Humboldt” right.

The speeds, for anyone keeping score:

Mode	Speed	Memory
Text only	152 tok/s	17.9 GB
With an image	67.7 tok/s	22.1 GB
With audio	147 tok/s	21.0 GB

The whole thing lives in about 22 GB at its hungriest. That fits on a Mac you can buy today.

And to be clear about what that means: nothing about that cart screenshot or that audio clip left my desk. The model makes no network calls at all — there’s no API key, no meter running, no terms of service, and no company on the other end deciding whether I’m allowed to keep doing this tomorrow.

Two things that fell out along the way

Neither of these was the goal, and both are probably more useful than anything else here.

NVIDIA’s reference code produces NaN on batched audio. If you feed it two clips of different lengths at once, the shorter one comes back as garbage — not an error, just silent NaN poisoning that spreads through the layers. It’s a masking detail: fully-padded rows go to negative infinity, softmax turns that into NaN, and the NaN travels. My port masks differently and stays finite. I want to be careful here — this is one specific path, and I could be wrong about how much it matters in NVIDIA’s own pipeline. But it reproduces on their code, not just mine.

The vision tower doesn’t normalize its own input. There’s a normalization layer sitting right there in the checkpoint, and it’s dead weight — NVIDIA switches it off and expects whatever calls it to do that job. Feed it raw pixels like a reasonable person would and you get plausible-looking garbage, silently. This one cost real time and it’s the trap anyone else attempting this will hit.

And a third, for the truly nerdy: several settings in the config file are lies. Not maliciously — they’re just vestigial, left over from an earlier design, and the live code ignores them completely. If you build from the config instead of tracing what actually runs, you’ll produce something that looks correct and isn’t. I only caught it by following the real code path line by line.

So who does this actually help?

Fair question, and I want to answer it honestly instead of waving at the future.

The narrow answer: it saves the next person about a week. Right now, anyone with a Mac who downloads this model reads that line on the model card — “requires a multimodal runtime that implements the C-RADIO ViT-H and Parakeet Conformer forward passes” — and that’s the end of the road. It’s not a warning, it’s a wall. It means go build it yourself, and most people, reasonably, close the tab. Now they don’t have to. Clone it, run it, done. And the three traps I hit are written down, because the normalization one in particular would cost someone a full day of wondering why their model got quietly dumber instead of visibly broken. That’s the entire contribution: one person spent the afternoon so nobody else has to spend the week.

I’m not going to pretend that’s a huge number of people. Might be a few hundred. Might be twelve. That’s fine — twelve people not wasting a week each is still worth an afternoon.

The wider answer is the one I actually care about. I do private-AI work for firms that handle other people’s confidential material — lawyers, medical practices, accountants. The whole pitch is that the machine doing the reading is the machine on your desk, because a federal court already ruled that work you hand to a public AI isn’t privileged.

Until now, “on your desk” meant text only. If a client sends a photograph of a contract, or a recorded call, or a scan — the private option had nothing to say. You either sent it to somebody’s cloud and lost the privilege, or you did it by hand.

That changed today, on my laptop. A model that can look at the scanned page and listen to the recording, on the machine sitting in front of them, is not a small difference for those people. It’s the difference between a tool they can use and a tool they legally can’t.

And the honest third reason: the gap between “the weights are public” and “you can actually use this” is where most open AI quietly dies. Everyone celebrates the release. Far fewer people do the boring work of making the thing run somewhere real. That gap is usually not a research problem — it’s a few hundred lines nobody got around to writing. This one was maybe 90 KB of Python and an afternoon.

That’s the part I’d like more people to see. Not that I did something clever — I didn’t, I transcribed NVIDIA’s own math into a different framework and checked my work. But the wall between an open model and a working model is thinner than it looks, and it stays up mostly because everyone assumes someone else will knock it down.

The code is on GitHub, MIT, with every parity test in it. Don’t take my word for any number above — clone it and run the tests on your own Mac.

Credit where it’s due

NVIDIA built the model and released the weights and reference code openly. None of this happens otherwise.
yayr at mlx-community did the 4-bit MLX quantization I built on top of. Go give that upload a like — it deserves more than the zero it has.
Apple’s MLX team — and whoever added nemotron_h to mlx-lm, which is why the brain needed nothing from me at all.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

The Day Local AI Caught the Cloud: ds4, DeepSeek V4 Flash, and What Just Changed for Devs

Matt Macosko — Thu, 23 Jul 2026 18:30:07 +0000

If you write code for a living and you’ve been watching the local-AI space, May 9, 2026 is the date to circle. Salvatore Sanfilippo (yes, the guy who wrote Redis) shipped ds4 — a few thousand lines of hand-written C with Metal compute kernels, built for exactly one model: DeepSeek V4 Flash.

I ran the same prompt through three engines on the same 128 GB MacBook Pro:

DeepSeek V4 Flash via ds4 — fully local, off-cloud
Cloud Claude through my Max plan
Gemma 4 31B via MLX, also local

Local DeepSeek beat cloud Claude on wall-clock time. That sentence used to be science fiction.

▶ Watch the companion video — three engines, one prompt, three completely different aurora animations rendered in real time on the same machine.

The benchmark, for people who don’t want filler

Engine	Time	Output	Where it ran
DeepSeek V4 Flash (`ds4` local)	103 s	3,259 tokens	Apple Silicon GPU
Cloud Claude (Max plan)	192 s	~3,500 tokens	Anthropic data center
Gemma 4 31B (MLX local)	131 s	1,992 tokens	Apple Silicon GPU

The prompt was a single creative HTML task: “Build an animated northern lights scene — single file, vanilla JS, mountains, pine trees, twinkling stars, flowing aurora bands.”

Each engine produced a completely different aurora. None of them hit the network during inference. (Yes, I checked with lsof. Yes, this is the same lsof audit pattern from the AirGap NDA piece.)

Three architectural decisions in `ds4` worth understanding

This is the part that matters if you’re a developer thinking about local AI infrastructure.

1. Asymmetric 2-bit quantization (only where quality is forgiving)

The naive approach to quantization treats every weight the same. ds4 doesn’t. Only the routed Mixture-of-Experts experts get compressed to 2-bit (specifically IQ2_XXS for up/gate, Q2_K for down). Every quality-critical path — shared experts, attention projections, routing, output head — stays at higher precision (Q8 or full).

Those routed experts are about 90% of the weight footprint. The other 10% is where small precision losses cause big accuracy losses. Quantize the 90%, leave the 10%, and you get an 81 GB file that still calls tools cleanly and writes coherent code.

This is the kind of tradeoff that only makes sense if you’ve stared at a specific model’s loss landscape long enough to know which weights tolerate compression. It’s a model-specific engineering decision dressed as a quantization recipe.

2. KV cache moved to disk (in 2026 SSDs are fast enough)

The “KV cache must live in RAM” assumption is from 2023. Modern Apple SSDs do 5+ GB/s sequential reads. ds4 writes session state to disk and reuses it across runs, keyed by SHA1 of token IDs.

The practical effect: when Claude Code sends its 25k-token system prompt, that prefill happens exactly once, ever. Every subsequent session — including totally different agent runs that happen to share that prefix — reads from disk in milliseconds instead of recomputing from token zero.

If you’ve used long-context models locally, you know prefill is the slowest thing in the loop. ds4 makes it free after the first hit. That’s the kind of “small change, huge implication” move that took years to normalize. (See also: the disk-KV section in the ds4 README.)

3. Pure Metal, not CUDA-with-a-shim

There’s no PyTorch, no TensorFlow, no llama.cpp wrapper layer in the hot path. The compute kernels under metal/*.metal are written specifically for this one model on this one architecture. The acknowledgments thank llama.cpp and GGML — ds4 borrows quant layouts and select kernels — but it’s not a fork.

This narrowness is the point. Generic frameworks pay a tax for being generic. When you commit to one model on one chip, you can hand-tune away that tax. ~27 tok/s on an M3 Max 128 GB. ~32 tok/s on M5 Max. For agent loops on a laptop, that’s plenty.

Why this matters for compliance-sensitive devs

The same week, I’m still maintaining AirGap AI — a wi-fi-off, lsof-audited workflow for analyzing privileged documents (NDAs, client files, PHI, etc.) on a laptop with no outbound connections. Until last week, that was a Llama 3.3 70B story. The capability ceiling was real.

ds4 raises that ceiling materially:

1M-token context — entire codebases, full deposition transcripts, complete contract sets, all in-memory in a single conversation
Quasi-frontier reasoning — if you’ve used Claude Sonnet or Opus, DeepSeek V4 Flash sits in the same neighborhood for most agentic tasks
Tool calling that works — Antirez tested it under coding agents (opencode, Pi, Claude Code) and the tool calls land reliably

For law firms, medical practices, and compliance-bound shops, the math just changed. You don’t have to choose between “frontier-grade reasoning” and “data never leaves the building.” The hardware exists, the engine exists, the model exists, and the integration with Claude Code exists.

(If you’re trying to get a bar-association-defensible AI workflow off the ground, the AirGap landing page is where I keep my notes. The ds4 stack is going in there next week.)

How to actually run it

The full stack, all open-source:

# 1. Build the engine (Apple Silicon with Metal)
git clone https://github.com/antirez/ds4
cd ds4 && make

# 2. Pull the q2 weights (~81 GB)
./download_model.sh q2

# 3. Boot the local Anthropic-compatible server
./ds4-server --ctx 200000 --kv-disk-dir ~/Library/Caches/ds4-kv \
             --kv-disk-space-mb 16384

# 4. Point Claude Code at it
ANTHROPIC_BASE_URL=http://127.0.0.1:8000 \
ANTHROPIC_AUTH_TOKEN=dsv4-local \
ANTHROPIC_MODEL=deepseek-v4-flash \
claude

Or just clone nicedreamzapp/claude-code-local — DeepSeek V4 Flash is now the fourth fighter in the lineup, with a claude-ds4 wrapper that handles all of the above for you.

What this slots into

This isn’t a one-off. It’s the next click in a longer arc I’ve been writing about:

Three Generations of Running Claude Code Locally on a MacBook — What I Actually Learned — the long path from “barely works” to “actually replaces my cloud usage”
Cloud AI Coding Costs Keep Climbing — How to Pay $0 and Still Use Claude Code — the economic angle, before ds4
Pulling 10x My Subscription Value Out of Claude — what the cloud math actually looks like
What It’s Actually Like to Code By Voice — With the AI Replying In My Own Cloned Voice — the voice loop these models now plug into
Your Medical Practice Is Probably Using Cloud AI on PHI Right Now — why on-device matters for healthcare
If Your Law Firm Is Using Cloud AI on Client Files, You Probably Have a Problem — the legal angle
A Field Guide to Ambient Computing — the bigger frame this all sits inside

ds4 is the engine that finally makes the local-first version of all of those usable for production work. The local agent doesn’t have to pick which workload it’s good at anymore.

Where to follow

🛠️ github.com/nicedreamzapp/claude-code-local — the lineup, launchers, and benchmarks
🐳 github.com/antirez/ds4 — the engine itself
🌿 marijuanaunion.com — the broader writing on local AI, voice, and ambient computing
🔒 nicedreamzwholesale.com/airgap — the compliance-grade workflow notes
💬 Discord — NiceDreamzApps server

May 9, 2026. The day a single C file caught up to the data centers.

This is the technical companion to the headline piece on Marijuana Union. Companion video: youtu.be/7l8-s8xkpms. For local-AI consulting on compliance-sensitive workloads, see AirGap AI.

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

The IRS Said Yes — and What That Does and Doesn’t Mean

Matt Macosko — Thu, 16 Jul 2026 06:10:44 +0000

Back on June 3rd I published a piece here saying the Cannabis Device Safety Institute had filed its federal application, and I made a point of leaning on that word — application. Filed, not granted. Pending, not approved. Not a tax-exempt charity, contributions not deductible, and we’d say so plainly every time, because saying it any other way would be exactly the kind of thing this institute exists to push against.

So I owe you the update in the same register.

The IRS granted it. The determination letter is dated June 29, 2026. CDSI is exempt from federal income tax under Section 501(c)(3), classified as a public charity under Section 509(a)(2) — not a private foundation — with the exemption effective April 27, 2026, retroactive to the day the articles were filed. Contributions are deductible under Section 170.

That’s the whole announcement. Now let me do the part I think matters more, which is being precise about what it does and doesn’t mean.

What it doesn’t mean

It is not an endorsement. Recognition of exemption is a determination about tax status. The IRS did not review our methodology, did not evaluate our off-gas protocol, and has no opinion whatsoever about whether ceramic donut atomizers off-gas at 650°F. No agency has blessed CDSI’s findings, because CDSI hasn’t published findings yet. If you ever see me imply otherwise, call me on it.

It doesn’t make CDSI a regulator. We’re a private nonprofit. We have no authority over anyone, we can’t compel any manufacturer to do anything, and we’re not seeking a government designation. The model is UL, ASTM, the NFPA Research Foundation — bodies that earned standing by being useful and rigorous, not by being appointed.

It doesn’t mean we have a lab — yet. Right now CDSI pays accredited independent labs and publishes what comes back. Pay the lab, not pay to pass. But the goal was never to outsource forever: the plan is for this institute to build and run its own testing bench, because the body that writes the methodology should eventually be able to execute it too — and the determination letter is exactly what makes that fundable. Foundation grants and tax-deductible donations can now go toward standing up a lab of our own. If and when that lab exists, nothing about the discipline changes — open methodology, public reports, every conflict on the cover page.

It doesn’t finish the paperwork. California still treats CDSI as a taxable corporation until a separate state filing goes through. That one’s in an envelope, not a press release.

About the timeline, honestly

We filed the full Form 1023 on June 3. The determination is dated June 29 — 26 days later.

The IRS publishes exactly one number about how long this takes: “We issue 80% of Form 1023 application determinations within 191 days.” That’s from their own “Where’s my application” page. So: 26 days, against a published benchmark of 191, on the long form rather than the 1023-EZ shortcut, with no request for expedited handling.

I want to be careful with that number, because it would be very easy to turn it into something it isn’t.

I can’t tell you it’s a record. Nobody can tell you that, about any organization. The IRS’s public files record determination dates by month only — no day — and carry no application-submitted date at all. The interval literally cannot be computed from public data, including the IRS’s own. So anyone claiming a record in this category is claiming something no dataset could check, and I’d rather not be that guy.

I also can’t tell you the speed proves the application was good. That’s the flattering read, and I don’t think it survives scrutiny. The IRS has fast-track lanes for straightforward cases, and how big those lanes are isn’t published. The six days between answering their follow-up letter and getting the determination is, as far as I can tell, just the IRS’s own internal rule about how fast a specialist has to close a case once you respond. And the biggest variable — when the application got assigned to a human at all — was completely outside my control and I can’t explain why it happened when it did.

What I can say is what we actually did, and let you decide if any of it mattered:

We paid $600 for the long form instead of $275 for the EZ, on purpose, because the EZ doesn’t have room to explain why a fee-for-service testing subsidiary serves a public mission — and filings like ours get bounced back to the long form anyway, months later. The slow-looking choice was the fast one.

We disclosed the conflicts instead of burying them. The founder of a cannabis hardware standards body builds cannabis hardware. That’s on the record, in the conflict of interest policy, in the state charity filing, and it’ll be on the cover page of every report we publish.

We went through this site and cut every claim we couldn’t prove — before we filed, not after. No “first,” no “leading,” no partnerships that hadn’t happened. Reviewers read your website. There was nothing to pick at because we’d already picked at it ourselves.

And when the IRS wrote asking for more information, we answered within hours of opening the envelope instead of sitting on the 28 days we had.

None of that is clever. It’s just doing the boring things in the right order.

What it actually changes

Donations to CDSI are now tax-deductible. That’s real — it means a foundation can fund hardware testing, and an individual who cares about this can help and take the deduction.

CDSI is registered with the federal government and eligible for federal grants — that happened back on June 23, with the SAM.gov registration going active.

Put those together and here’s the thing I care about: the money to characterize this hardware can now exist. For fourteen years the reason nobody tested the device was that no one would pay for it. In 2016 I paid a lab out of pocket to run an off-gas test on a concentrate vaporizer because there was no funding source in the world for that work. That’s still the founding artifact of this institute, and it’s still the reason it exists.

Now there’s a vehicle that can receive that money. That’s what a determination letter is. Not a trophy — plumbing.

Where this goes

The next thing I want to make the case for is bigger than safety, and I’ll write it properly soon.

It’s this: you cannot honestly answer whether inhaled cannabis helps someone until you know what the device contributed to the dose. Heat a vaporizer and it puts its own chemistry into the same stream carrying the cannabis. So every measured outcome — a contaminant, a symptom, a relief — has two possible sources: the material and the machine. That’s not noise you can fix with more study subjects. It’s an attribution problem, and no sample size touches it.

Every other measurement science solved this a century ago. The chemist runs a blank before running samples. Nobody doses a patient through an uncharacterized nebulizer. Cannabis research does the equivalent constantly, not out of carelessness, but because no one was ever responsible for the device.

Which means hardware characterization isn’t adjacent to the medical question. It’s upstream of it. First you characterize the device. Then you have a device you can trust as an instrument. Then — and only then — you can ask what the medicine does to a person and believe the answer.

It all starts with the devices. Not because the devices matter most. The person matters most. But the device is the part you have to understand first in order to understand any of the rest of it honestly.

That’s the work. The letter just means we can afford to do it.

The Cannabis Device Safety Institute is a California nonprofit public benefit corporation, recognized by the IRS as exempt under IRC § 501(c)(3) and classified as a public charity under § 509(a)(2) (determination letter dated June 29, 2026; EIN 42-2429365). Recognition of exemption is a determination of federal tax status and is not an endorsement of the Institute or its findings by the IRS or any government agency. cdsi.click

Originally published at Marijuana Union. The Cannabis Device Safety Institute is an independent 501(c)(3) nonprofit standards body for cannabis consumption hardware — open methodology, public reports, every conflict of interest on the cover page. Methodology, papers, and the public record: cdsi.click.

M5 Max + 128GB = a 30B AI Coding Agent Running Locally. Wi-Fi Off.

Matt Macosko — Mon, 29 Jun 2026 17:18:47 +0000

The M5 Max MacBook Pro with 128 GB of unified memory is the first laptop that can hold a frontier-class coding agent entirely in RAM. No GPU rack. No cloud. No subscription.

That clip up top isn’t a render. That’s Qwen 3 Coder — 30 billion parameters, 8-bit MLX — running on this MacBook with the Wi-Fi off. Around 55 tokens per second. Total cost to keep running it: zero.

The thing that matters more than the spec sheet is what it actually unlocks.

Why the M5 Max changes the math

Until now, running a 30B+ parameter model meant a GPU rack — or paying a cloud API per token. The M5 Max changes that:

128 GB unified memory. The entire model lives in fast RAM. No GPU offload, no quantization tricks past 8-bit. The CPU and “GPU” share the same memory, so there’s no copy step between them.
Mixture-of-experts plays perfectly with Apple Silicon. Qwen 3 Coder is 30B total but only 3B active per token. That’s a math problem the M5 Max’s memory bandwidth eats for breakfast.
MLX runs at near-CUDA speed. Apple’s native ML framework hits ~55 tok/s on the 8-bit quant. No CUDA tax, no Nvidia driver politics, no $40,000 GPU bill.
It’s a regular store-bought laptop. No GPU rack. No data center. No cloud bill. You can run it on a plane.

This wasn’t possible on a laptop a year ago. It is now.

What you can do with it now

Read a legal contract — and have it never leave your machine.

Most AI tools pipe your document to a server somewhere. With this setup, the bytes don’t leave the laptop. NDAs, supplier agreements, employment contracts — review them at your kitchen table without uploading them to anyone.

Write production code in a couple of seconds.

The video shows it: real Python function, real Qwen output, no edits. The agent’s tool-calling is good enough to drop into Claude Code’s loop, where it’ll edit files, run shell commands, and iterate. It’s plenty for everything from one-off scripts to refactoring real production code.

Analyze patient charts without a HIPAA violation.

For doctors, therapists, intake clinics — anything with PHI on it — local-only AI isn’t a nice-to-have, it’s the only legal option. Same model, same speed, zero bytes leaving the device.

Build agents that don’t charge you per call.

This is the one most people sleep on. Pay-per-token cloud APIs make agents expensive to leave running. Once the model is local, you can let an agent loop overnight, hit it with thousands of requests, kick off a watcher that scans your inbox every two minutes — and the cost stays at zero.

The full stack

Here’s the receipt:

Hardware: M5 Max MacBook Pro, 128 GB unified memory.
Model: Qwen3-Coder-30B-A3B-Instruct-MLX-8bit — about 30 GB on disk. Mixture-of-experts, ~3B params active per token.
Server: A small Python proxy at localhost:4000 that speaks the Anthropic Messages API, so the Claude Code CLI thinks it’s talking to the cloud — except it’s talking to a hard drive.
Total monthly cost: $0 once it’s downloaded.

That’s it. No Docker, no Kubernetes, no VPS. Just a laptop on a desk.

The performance, honestly

The local-AI space is full of overclaims, so the straight numbers:

55 tokens per second on a real coding task. Sustained, not peak.
Two seconds to write a working find_median() function. Three to four seconds for most refactors.
Tool-calling reliability is good enough for the Claude Code agentic loop. Not as locked-in as Sonnet 4.6, but plenty for getting work done.
What it’s not: a Sonnet replacement for nuanced reasoning, long contexts, or really tricky debugging. For day-to-day code agent work, it more than holds its own.

Why the offline part matters

The reason “Wi-Fi off” keeps coming back in the demo isn’t a gimmick. It’s the whole thesis.

If a tool needs the internet, three things are true:

Someone else can read what you sent.
Someone else can charge you for it.
Someone else can take it away.

If the same tool runs locally, none of those are true. That’s a different category of software. Not better at every task — but yours.

Benchmarks — actually run, not cited

Big claims need numbers. So here’s what Qwen 3 Coder 30B-A3B (8-bit MLX) actually scores on this MacBook, run end-to-end against the local localhost:4000 server. Every problem solved by the model, executed in a Python subprocess, scored pass/fail. Pass@1, temperature=0, single sample per problem.

Benchmark	N	Pass@1	Notes
HumanEval	164/164 (full)	81.7%	Python function-completion classic. Saturated benchmark; modern coding models cluster 75–95%. 14 min total wall-clock.
MBPP (sanitized)	168/427 (sampled)	83.3%	Mostly Basic Python Problems. Pass rate was stable since n=120; a few outlier tasks induce very long model responses, so I cut off at 168.

Both runs used pass@1, temperature=0, 10s execution timeout, on the local 8-bit MLX quantization. No retries. No best-of-N tricks. Single sample per problem.

For context — what the bigger sibling scores on harder benchmarks

The Qwen team didn’t publish HumanEval/MBPP for any Qwen3-Coder variant — they consider those benchmarks saturated. Their official benchmarks are agentic, and they ran them on the flagship Qwen3-Coder-480B-A35B-Instruct (the bigger sibling, ~16× the active params of the 30B-A3B running on this laptop). For context — here’s what the flagship 480B scores on those harder agentic benchmarks compared to the major closed models:

Agentic Benchmark	Qwen3-Coder 480B	Claude Sonnet 4	GPT-4.1	DeepSeek-V3
SWE-bench Verified (500-turn)	69.6	70.4	—	—
Terminal-Bench	37.5	35.5	25.3	2.5
BFCL-v3 (function calling)	68.7	73.3	62.9	64.7
Aider-Polyglot	61.8	56.4	52.4	56.9
WebArena	49.9	51.1	44.3	40.0

Source: Qwen team’s official blog. The 30B-A3B running on this MacBook is a smaller sibling of the 480B — it trades absolute peak agentic ceiling for fitting in 30 GB and running 24/7 on local hardware. For most coding tasks people actually do in a day, HumanEval/MBPP-class accuracy matters more than the SWE-bench top-line, and on those it sits where it should: useful, fast, local.

Where this is heading

The next year of the AI conversation isn’t going to be “which model is smartest.” It’s going to be “which workloads belong on your machine, and which belong on someone else’s.”

Compliance-bound work — legal, medical, financial — is going to move local fast. Code-agent loops will follow because the math (per-call cost vs. zero) is brutal. The M5 Max with 128 GB of unified memory is the laptop that lets that happen.

Try it yourself

The launchers are open source on GitHub: nicedreamzapp/claude-code-local. The README walks through downloading the model and pointing Claude Code at the local server.

For law firms, medical practices, and accountants that want help getting this running on their own hardware — that’s what AirGap is. 14-day pilot, fixed scope, the data never leaves your machines.

— matt

Originally published at Nice Dreamz Wholesale. Run AI locally on your own hardware with claude-code-local, open source and no cloud required. More at nicedreamzwholesale.com/software.

DEV Community: Matt Macosko

A Federal Order Switched Off Anthropic’s Best AI Overnight — and Made the Case for Private AI

🗓️The timeline

🏛️What the government actually ordered

🔓The jailbreak: a real event with an inflated headline

🕵️Who is Pliny the Liberator?

🎭The strange part: a jailbreaker can be a lab’s best marketing

🌊And he is far from alone — the scene is moving fast

🛡️Anthropic’s side — and it’s a strong one

📉The scandal underneath the scandal: silent downgrades

👁️The quieter fear: are they watching, and training on what you tell them?

🧵So why did people on X think it was all because of Pliny?

🧭The lesson, from where I sit

📌The bottom line

Why I Quantize Open-Weight Models for Macs — And Why Your Law Firm Should Care

Why I Quantize Open-Weight Models for Macs — And Why Your Law Firm Should Care

💻The hobbyist case is easy

⚖️The interesting case is law firms

🔍Why open weights specifically

🧪What you can do with this

🚫What this is not

🔗Links

Whose work this stands on

Two AI Agents, One Browser Window: What Actually Breaks, and What Doesn’t

🚨The symptoms

🙈The wrong conclusion

🔀What’s actually going on

🛠️Fixing routing

🔦The focus bug is real, and already fixed

🔒The fix people reach for that doesn’t work

✅What I’d tell you to do

🧪Three tests, so you don’t have to take my word for it

What I did not test

Whose work this stands on

504,571 Brain Cells, 4 Labs, One Hypothesis: A Citizen Pass at Parkinson’s

504,571 Brain Cells, 4 Labs, One Hypothesis: A Citizen Pass at Parkinson’s

💡Where the idea actually came from

🧪What I actually ran

💊Why this could actually matter

⚠️What I want to be very careful not to claim

📖Why I am publishing this

Whose work this stands on

I Added Day-One Muse Glimmer Support to Apple MLX-LM

🔩the handful of things this model does differently

🔬i wanted proof, not looks fine

🪜someone builds the ladder, everybody climbs

🤗the weights are up if you want them

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

🧰The setup

🐛The bug

🔧The fix

🧠But did removing the big harness make it dumber?

📦Where it all lives

My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.

The setup

The bug

The fix

But did removing the big harness make it dumber?

Where it all lives

I wrote the missing Apple Silicon runtime for NVIDIA's Nemotron Omni

Verified, not asserted

Speed and memory

Why bother doing this locally

Try it

NVIDIA Shipped a Model That Sees and Hears — It Just Didn’t Run on a Mac. So I Wrote the Missing Piece.

What Nemotron Omni actually is

The part I keep having to relearn

Proving it, instead of vibing it

So does it actually work

Two things that fell out along the way

So who does this actually help?

Credit where it’s due

The Day Local AI Caught the Cloud: ds4, DeepSeek V4 Flash, and What Just Changed for Devs

The benchmark, for people who don’t want filler

Three architectural decisions in ds4 worth understanding

1. Asymmetric 2-bit quantization (only where quality is forgiving)

2. KV cache moved to disk (in 2026 SSDs are fast enough)

3. Pure Metal, not CUDA-with-a-shim

Why this matters for compliance-sensitive devs

How to actually run it

Three architectural decisions in `ds4` worth understanding