DEV Community: Dragos Roua

[ Video ] Stories from the Vibe Coding Trenches – Accidentally Overpaid $150 to Cloudflare

Dragos Roua — Wed, 14 Oct 2026 12:55:49 +0000

I’m starting a new series of videos: stories from the vibe coding trenches, with real life situations encountered during my vibe coding endeavors. As you already know, I have 15 apps in the App Store already, and I started all this in February this year.

The first story is about how I overpaid $150 to Cloudflare, because of a rookie mistake. Full video is at the top of the article.

I launched Flight Lens because I wanted to monitor air disruptions and delays – I still travel a lot – plus a simple way to track ticket prices. I use a relatively expensive API for the airport / airborne planes data and the disruption engine is completely proprietary. It takes flight delay data, planes in the air, announced delays at takeoff and landing, and so on, and turns that into crystal clear, actionable information. Almost all the app work went into making that engine efficient enough to stay inside the paid API’s beginner plan.

The API itself was about $45 a month. The backend was deployed on Cloudflare for the usual $5. After almost 2 months of back and forth I launced and everything looked fine. People were using the app and the feedback was really nice.

Then the Cloudflare invoice arrived: $150.

I’ll do the math for your: that is three times the API cost. I literally had no idea what hit me. But then I drilled down the Cloudflare’s billing dashboard, and found out that database reads and database writes are separate units. While I was heavily optimizing the API side of the disruption engine, I had written roughly 150 million times in a month. I was nowhere near the read limits, but the writes had gone way over.

So, I went back in and optimized again, this time watching both the API limits and the database write ceiling. I also set up billing alerts so I get notified before anything climbs too far (in all fairness, those alerts should have been there from the start).

Even with more than thirty years of writing code, these newer services still surprise me – especially if skip how they actually charge. The takeaway here is “just pay attention”, nobody will warn you if you ht some obscure paying ceiling.

The next stories in this series will be lighter, this one was the only expensive one (so far).

If you like this kind of content, make sure to like, share and subscribe to my YouTube channel.

[ Video ] Running Liquid AI LFM V2.5 on my 16 GB MacBookPro

Dragos Roua — Thu, 08 Oct 2026 08:57:53 +0000

This is our fourth video in the series: Local AI – models that you can actually run on an entry level machine, like a 16GB MacBookPro.

This time we’re testing LFM 2.5 3B, a 3 billion model from Liquid AI, the US neolab that started at MIT.

The testing methodology is the same: for text generation, we’re asking “what is a dolphin?” and for code generation we’re asking for a small utility in Python that can count words in files (and in multiple folders, if asked).

The text generation is very fast and the answer is more than reasonable. No hallucination and good prompt adherence.

Coding is decent. The model spent about 2 minutes on the task and delivered a working Python file. It also asked for tools right after the prompt was processed, and it went on searching the web first. After that, it generated the Python utlity on disk.

All in all, this looks like a solid model. What makes it even more interesting is the fact that it is marketed as an OCR and document recognition model. I didn’t test those features yet, maybe in a future video.

For newcomers: the actual screen recordings are not sped up, or edited in any way, because I want my subscribers to have a real idea about what it is to run those models on their hardware.

If you like this, like, share and subscribe.

If you want other models tested, drop a comment, I read them all.

For more details about how I set up the test environment, as well as the list of the previous tests, check out this playlist: https://www.youtube.com/playlist?list=PLYoyB8nx4G30

[ Video ] Artificial Intelligence or Super Intelligence?

Dragos Roua — Wed, 07 Oct 2026 01:12:53 +0000

The internet was boiling again last week, over a new approach towards AI: from now on, we’re not calling it Artificial Intelligence anymore. It will be SI, “Super Intelligence”.

Seen from a distance, this looks like yet another internet shallow conversation. But I think it’s worth staying with it for a while, so I made a short video to explain why. Watch it above if you’re the visual type.

If not, keep reading for a short description.

I think there are 3 reasons this is important.

Machines, Politics and Competition

First, it’s an evolution from the “machine” to a broader understanding of the “intelligence” term. We started with “machine learning”, we evolved to “artificial intelligence” and now we are trying to move to “super intelligence”, which drops the “machine” layer completely. I don’t quite agree with this. Intelligence, as we know it, is still human-generated and empowering some files on our computers with this feature will just make it easier to anthropomorphize them. Which is obviously weird and dangerous.

Second, this looks like a political bow to the higher ups. President Trump held a poll on his Twitter account, asking what should be the name used going forward, and “Super Intelligence” won. Still unclear why we cannot just keep using “Artificial Intelligence”. Anyway, AI overlords (or should we call them SI overlords going forward?) caved in, so Elon Musk announced he will change SpaceXAI to SpaceXSI. As a political move, agreeing openly with the president can grant you some benefits. But it looks weird.

And third, if the term really catches up, the one that names a category usually gets a bit of a head start. Elon Musk has a long standing beef with Open AI, which he founded, by the way, so obsoleting the “AI” in the name will put Sam Altman at a slight disadvantage.

All in all, we’re looking at an interesting marketing move. One that looks shallow, but could shape the names we’ll be using in the next 2-3 years.

If you have any comments about this, please do so under the video, I read them all. And if you like this kind of content, like, share and subscribe!

[ Video ] How To Set Up Local AI on a 16GB MacBook

Dragos Roua — Sat, 03 Oct 2026 00:18:31 +0000

One of my YouTube subscribers asked, in a comment, how to set up a 16GB M1 MacBook Pro so it runs models in a more comfortable way, not necessarily in the terminal. This came as a reaction to my local models demo videos, which seem to be picking up lately. People enjoy that content, but they want something more visual to play with them.

So, I made a new video, specifically for this. Just watch it above, if you’re the visual type.

If not, read on for a detailed breakdown on how it’s done.

Setting Up The Local AI Foundation on a MacBook

The basic understanding of running local AI is that you first have to “serve” the model, and that is done usually in the terminal. After this step, you can start “consuming” the model, via a web interface, or even a desktop app.

Serving the model is done, on a mac, with an utility called mlx-serve. It’s a solid command line utility, with a lot of arguments, which sometimes may be overwhelming. But the TLDR here is that you only need to remember 2 commands:

mlx-serve run

mlx-serve serve

Depending on whether or not a model is already local, it also downloads the model on your machine. We are not using ollama or other more established binaries because mlx is specifically optimized for the Metal architecture of your Mac. Simply put, the models will run faster and with less resources.

Serving the model is already 50% of the job.

From there on, I explore 3 ways to consume this.

Using Local AI With Grok Build

Grok Build is my default CLI these days – I’m vintage, I know, and I spend 90% of my time in the terminal. I already talked about how to set up free models (both local and remote, on OpenRouter) on this post. The video is just a very short version of that.

The TLDR is that you will edit Grok’s config.toml by adding the model characteristics, and then you’re just picking the model mid-session with /models inside Grok Build. It’s not complicated, if you like staying the terminal, and, like I said, that’s my goto approach when using AI in general.

Using Local AI With OpenWeb UI

OpenWeb UI is a chat wrapper on top of the local model, but it offers a relatively rich experience. It’s very simple to install, and it will offers you a familiar, ChatGPT style interface. You can switch models or adjust the model parameters, and generally do whatever you do with a standard ChatGPT interface, only it uses a local model. Use it if you’re spending most of the time in the browser.

Using Local AI With MLX-Serve (the desktop app)

The most complex and rich experience comes from MLX-Serve, which is a notarized by Apple app, built on top of the command line utility mentioned at the beginning of this article. It is by far the nicest, most elegant way to run local AI on your machine and it has all the pre-requisites to become a more established app for day to day AI tasks. On top of the chat experience, you also get to generate images, voice (text to speech), music and even videos. There’s solid support for defining agents too, and you get recurring tasks as well.

That’s my recommendation for anyone who want to try local AI on a Mac.

All the links for the above programs are in the video, feel free to check it out.

As usual, if you like this kind of content, like, share and subscribe to my YouTube channel, you will help local AI grow.

What Makes an AI Model a “Really Good” AI Model?

Dragos Roua — Fri, 02 Oct 2026 00:21:03 +0000

The pace at which neolabs are launching AI models is getting ridiculous. Every week is “the best week in AI”. Every few days a new “really good AI model” is launched. Every few days a new iteration on that pops up and every couple of weeks an entirely new class of models is available.

It’s becoming hard to tell what model is actually the best one. Or, put it more bluntly: what good is this AI for.

If you want to see this rant as a video, just click the image cover.

If you prefer reading, keep going.

Honestly, I don’t believe the hype. I don’t think everything is being replaced at the speed described. I think we are getting hit by a continuous flux of information from three sources, and those three sources account for most of the new-stuff stress.

paid shills
neolabs PR departments
red-pilled AI people

First, the paid shills. I’ve seen this many times in crypto. These are just influencers for hire, getting paid to pass the neolabs messages over.

Second, the neolabs themselves. They keep pushing a narrative that gives the impression something important is happening, or it’s just about to happen. Like yet another OpenAI model “escaping confinement” and hacking some random site on the Internet.

Third, and this one is the most impactful, just normal people who got red-pilled by AI. They are almost always genuine. They truly believe “their” model is the best. Let’s not forget that we are, essentially, just apes with clothes, so tribalism is still running high in our blood.

The thing is Opus 5.5 and Grok 4.7 are almost identical. If there is a difference, it is half a percent on whatever general scale we pretend we have.

How We Make an AI Model a “Really Good AI Model” – And Even Pay for This in the Process

We came a long way from early ChatGPT 3.0, when there was a chat box and the thing on the other side was literally some software giving plausible answers – not a call center in India. That was a real breakthrough. It bended our usage patterns so we started to get more of it. We started to ignore how often it hallucinated, and we gave extra weight to the the times when it didn’t. We walked the path that looked promising, evolving, amplifying.

And that’s how the neolabs closed the loop. They looked where we’re heading, they took our feedback and trained toward our expectations. They literally did product discovery with us as the research panel, while we paid for the privilege.

You see, different clusters of users prompt their models for different problems, so neolabs track these patterns and find verticals inside a field that is literally endless.

For instance, Grok 4.6 is very good at coding. Most of the problems that ecosystem solves are related to coding. The user base around SpaceXAI is programmers.

Gemini is very good at summarization. They did AI snippets in search for so long, that they became really good at it. The model got optimized for a tight, targeted answer that can sit in the first position, above the actual results.

Claude noticed that a lot of their customers were designers and optimized for that. Then Anthropic launched Opus 5.5 and someone made a seed movie showing what a beautiful marketing movie that model can do. If that guy was paid or genuine, doesn’t matter. What matters is that thousands of red-pilled AI people followed. Suddenly the internet is full of similarly looking videos you can glance at and immediately recognize them as Opus 5.5.

And from that one use case, people infer Opus 5.5 is way better than Grok 4.7.

It isn’t. That’s how the models are trained and exposed.

The Real Moat Is the Question

If you know what to ask, you’re suddenly in a place where pretty much every model does the job. Large language models are just statistical approximation machines. Some are trained specifically for marketing movies, some for code, some for snippets. But at the end of the day they are more or less equal depending on the question.

For example, I have been writing code for 35 years. I know what to ask when it comes to coding. I do not know what to ask about marketing or movie-making, because I hardly honed my skills in those areas. But the coding hard skills, trained over decades, allow me keep my AI spending low, because I already know that, deep down, they are all the same machine.

If you know what to ask, AI is pretty much the same. Don’t get fooled into believing Opus 5.5 is in any way better than ChatGPT Astra, or Grok 4.7, or DeepSeek, or GLM, or Qwen.

[Video] Running Xiaomi MiMo V2.6 Qwen 9b Distill on a 16GB MBP

Dragos Roua — Wed, 30 Sep 2026 07:43:15 +0000

This is the third video already in the series I started this month. I am trying to share my experiments in running relatively small – but powerful – models on an entry level machine, my 16GB M1 MacBookPro. Last week I tested Google Gemma 4 E4B on the same 16GB M1 MacBookPro, and the week before that it was Ternary Bonsai 2, the 2-bit squeeze of Qwen 3.8 27B.

If you prefer reading, let’s go.

Nothing changed in the methodology, I’m still using the same questions and coding challenges, and the screen recordings are not sped up or edited in any way, so you get exactly what can you realistically run on an entry-level machine, at real speed – with the wifi off if you want it off.

Today’s model is Xiaomi’s MiMo V2.6, a 9 billion parameter distill of Qwen. Xiaomi took Qwen 9B, post-trained it on their own traces, and put it out as an agentic checkpoint. On paper it is better at long tasks and at coding that needs tools.

Fair warning that I changed one thing in the setup. In the Bonsai video I used a terminal. This time I ran everything inside MLX Core, a native Mac app that sits on top of MLX, link at the end of this blog post. It looks a little like ChatGPT or Codex Desktop, which makes the recording easier to follow. It also adds a bit of memory overhead, so the tokens come out a little slower than a raw CLI run.

The first prompt was: “what is a dolphin?”, and the answer came back in about 30 seconds, at roughly 12 tokens per second. The answer was fuller than Gemma’s and Bonsai’s. More species, more structure, it felt like a model that likes to finish the thought.

The second prompt is the small utility I keep using as a test: “write a Python script that counts the words in every file in a given folder”. MiMo did something the others did not: before it even wrote a line of code, it asked me to enable tools. It already knew it would save something on the disk. Then it thought for about four minutes and produced word_count.py. The script runs ok, but the printed output has a little garbage in it — a “mojibake in files” line that came out of nowhere — but the job got done.

I would put MiMo 2 between Gemma 4B and Ternary Bonsai 2. Maybe on the same shelf as Bonsai. A little slower, maybe. Good enough for summarization and for the kind of quick utility you can throw at a 9B model and expect to walk away with a file. I would not start a long agentic coding session on this machine with this checkpoint.

And then the part that still matters more than the tokens-per-second number: it is free, it does not bill you per word, and it keeps working when the laptop is offline.

If you have another model that you wanna see it tested on this same 16GB M1, leave a comment under the video. I already have the next three from the last round of comments.

A few links, if you want to run the same thing:

Model card: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
MLX Core / mlx-serve: https://github.com/ddalcu/mlx-serve
Second video in the series, Gemma 4 E4B
First video in the series, Ternary Bonsai 2

Thanks for watching and spreading the word. Every share makes local, sovereign AI a little less of a lab toy.

[Video] Running Google Gemma 4 E4B on my 16GB MacBookPro

Dragos Roua — Fri, 25 Sep 2026 02:27:22 +0000

Last week I started a series about running open weights models on your own hardware. The goal was to give my subscribers a good understanding of what it is to run local, private AI on an entry level computer. The first post went really well, with more than 3000 views so far and dozens of hours watched. So today I’m pushing things forward, with yet another test, Google Gemma 4B E4B.

This one has just 4 billion parameters and file size is roughly 5GB, so it fits way better than the Ternary Bonsai we tested in the first video. For a quick reminder about what amount of RAM you need to comfortably run open weights models, you can have a look at the episode 4 of my series about how to choose an open weights model.

For this test I chose 2 tasks. First one is my usual “what is a dolphin?” question, but second one is a simple code generation task. Google Gemma 4 E4B managed both very well. In terms of time, it was usable, the question was answered in a few seconds, while the code generation took about 2.5 minutes. Kind reminder that all recordings in these videos are not edited in any way, no sped up, nothing, so you can have a real understanding of what it means to run the tested model.

All in all, Google Gemma 4 E4B is a nice surprise. I would not use it for very intense code generation tasks, but for quick functions, or summarization, I think it’s an interesting choice. The fact that it only takes 5GB of memory, leaving a lot of room for other programs on my tiny 16GB M1 MacBookPro is a big plus too.

If you have any suggestions about which models should I test next, drop a comment on that video. An of course, if you like this kind of content: like, share and subscribe!

Thanks for watching and spreading the word, you’re helping local, sovereign AI to come a little bit closer with every share.

[Video] I Tested Grok 4.7 With One Simple Job: Turn My vi Task Files Into a Bash Agenda

Dragos Roua — Thu, 24 Sep 2026 02:04:31 +0000

Grok 4.7, from SpaceXAI, was released yesterday. It was delayed by almost 2 weeks, but it eventually came out, and I had a chance to test it. There’s a tiny video you can watch, if you want to jump straight to the action.

If you prefer reading, here we go.

For the last few months, I’ve spend the majority of my time in the terminal. That’s where I fire up my agents and where 90% of my work is done. Getting out of there just to log what I did started to feel cumbersome. So I started to write these very simple files, like thursday-24-09-2026 where I would just edit with vi the things I’ve did. Soon, I added a simple visual cue: “x” in front of it, for a task done, “-” for something still going on.

So, when Grok 4.7 arrived, I thought: “why not testing it by building a proper shell agenda, based on the format of my files?”.

45 minutes later, the agenda.sh file, was ready. Written in pure bash, self-contained, highly portable (can run everywhere bash runs, being it mac, windows or linux). The repo link is in the video description, and the license is Apache 2.0, which basically means you can start using it now, for free.

From testing Grok 4.7 itself, I got 3 main takeaways:

Better prompt adherence. It follows the instructions close and I find this positive: I don’t want the model to get creative when I already know the what I want to do.
More thorough. It circles around the task and tries to see it from more angles, and that translates into higher confidence when it hands the file over.
Greedy on tokens. I’m having a SuperGrok subscription, and I used high reasoning during testing, not high. I started the task at 14% of the weekly quota and ended at 54%. Forty percent of the week on forty-five minutes. Right now it is still heavy.

If you already tested Grok 4.7 on a real use cases, pleas leave a comment in the video, I check them all. And if you like the video, like share and subscribe!

[ Video ] A System One AI Doesn’t Talk, It Only Decides – And It’s Perfect for Assess Decide Do

Dragos Roua — Tue, 22 Sep 2026 04:51:41 +0000

Last week, a new AI model was released, and it’s nothing like the AIs we already have. It’s called “JEV”, it’s a System One AI, and it only decides, it doesn’t generate any text. I got early access to it and I found an interesting use case: making it the Decide layer of my Assess – Decide – Do framework.

I made a video in which I show exactly how it’s done, and why this new paradigm is opening many new possibilities, check it out.

If you prefer reading, here’s what happened.

JEV cannot output text, like I said, but it can answer questions, using 3 pre-defined primitives: noul (yes/no), choice (multiple options) and score (probabilities). I made a small system that builds a piece of software that you define only with one sentence. JEV picks that up, forwards it to Assess, where a LLM starts drafting the deatailed specs of what you want to build, including tests. Then it passes it back to JEV (classic ADD flow). If JEV has enough context, it passes the task forward to Do, where the LLM starts implementing. When implementation is done, it passes it back to Decide, where JEV runs the tests and decides whether or not the app is done.

In the video I made a small CLI in Python that counts the words in the files of a specific folder. But I reckon anything can be build this way. The most interesting part, at least for me, it’s the visual transition between realms, there’s something fascinating in the flashing of red, orange, green and white circles, as the action is moving between them.

I ran two tests of this system: the first one used the official JEV API, which is remote, as well as a solid LLM from OpenRouter, and it took under 2 minutes. The second one used a local implementation of JEV, and a local LLM, both way less performant, and that took almost half an hour. You can see in the video a sped up recording of that too.

I think this new paradigm of deciding with a structured output is opening a lot of new possibilities for AI. If you have any use cases, or just want more about this, please leave a comment on the video.

[Video] Running Qwen 3.8 27B Ternary Bonsai 2 on my 16GB Mac

Dragos Roua — Mon, 21 Sep 2026 02:59:07 +0000

Last week I started a new series of videos: local AI models, tested on my computer, at the actual speed, when possible. The goal is to give my subscribers a very good understanding of what it is to run local, private AI on an entry level computer.

The first one is a heavily quantized version of Qwen 3.8 27B. It’s called Ternary Bonsai 2, and it is roughly 7.5 GB on disk. The normal Qwen 3.8 27B is around 50GB so it won’t fit on my 16GB M1 MacBookPro. Not even close. If you watched my series about how to choose an open weights model, then you know that size of the model must be smaller than computer RAM. I explain all this in episode 4.

PrismML released Ternary Bonsai 2 ten hours before I recorded. I’ve been very enthusiastic at first, but after a couple more days of playing with it, I hit some limits, and realized that the team claims that the model retains 98% of the original Qwen performance must be taken with a grain of salt. I gave it a Three.js generation task and it worked for about 3 hours without producing relevant results.

Nevertheless, the model is still impressive. Especially when you think we couldn’t even dream about this level of performance a year ago.

Local, sovereign AI is evolving very fast. Once again, the video above is recorded on a 16GB M1 MacBookPro, not on a last generation Nvidia Blackbird, nor on a big memory Mac Studio. It’s just a tiny laptop.

I intend to make these series weekly, so if you have any suggestions about which models should I test next, drop a comment on that video. Of course, if you like this kind of content: like, share and subscribe!

Thanks for watching and spreading the word, you’re helping local, sovereign AI to come a little bit closer with every share.

The AI Agents Took Over My Blog Traffic. I Made Them Pay For That…

Dragos Roua — Sat, 19 Sep 2026 04:02:54 +0000

On a good day, Google sends to my blog about 20 people (clicks on the search results). On the same day, though, Google Search Console also shows me around 400-450 AI citations. That means agents consuming my content – either for responses in chat, or for training. The worst part? They never asked for permission, those bots. So I made AI agents pay for that.

If you’d rather see this as a video, just check out the link above.

If you prefer reading, go on.

How Did I Make Agents Pay?

The blog you’re reading right now at https://dragosroua.com is a static site. It’s built with astro, which pulls data from my old WordPress engine (I still add content there). Every build generates an .html file that’s served indiscriminately to everyone: humans, Google index bot, AI agents.

I added an extra step in the astro build process, that takes the content I’m getting from WordPress, and turns it into a stripped down markdown file, just the title, the publication date and the body. So now there are 2 files with the same slug, but 2 different suffixes:

blog-post.html (technically, this is blog-post/index.html, but that is not relevant for now

blog-post.md

Now, if an agent takes in an .html file, it still needs to do some parsing to get rid of the HTML tags, CSS, header metadata, javascript snippets, etc. This consumes some tokens. On the other side, if they just get the .md, they already have the content, no need for parsing.

On average, a model consumes 3x more tokens for HTML content than for markdown. On an expensive model (Opus, Astra, Grok) this may come up to $0.01.

So, I hid the .md file behind a x402 paywall (402 is actually part of the http specification, it was around since all the other codes: 404, 301, etc. Only recently there were actual implementations on the protocol, and the most popular one was launched and supported by Coinbase).

So what happens now, is that the agents are having a conundrum:

parse HTML and spend a variable (not known beforehand) amount of tokens
get the parsed .md and pay $0.01

The signal for this is given in the llms.txt file, which is a specification followed (in theory) by all agents. They read that file, and they understand how they can consume my site. Links inside llms.txt used to be all HTML, now they’re pointing specifically to the .md files.

Of course, not all of them are complying, and I’m not expecting all agents to pay now, but at least I’m trying. I’m sending the right signal, showing that the data is served on different levels, that it has some value, and that it comes with a bit of convenience – being already stripped down in a format AI agents understand.

What Open Weights Models Can You Realistically Run on a Medium Laptop (September 2026)?

Dragos Roua — Thu, 17 Sep 2026 02:51:15 +0000

Dario Amodei asked for a slowdown on artificial intelligence and a ban on open source AI development. The reaction was, luckily, overwhelmingly on the opposite side: Jensen Huang of Nvidia, and even heads of state, like Donald Trump and Xi Jinping, came out in favor of keeping AI development — and open source — moving. So, if open weights are staying, then a new question arises: which models can you run comfortably on a medium laptop in September 2026?

I travel a lot and I want intelligence on the go, on hardware I actually own.

This video above is my short answer to: what “open weights” really means, how much RAM you need, how quantization works, and my very own shortlist with models from Europe, the US, and China.

Watch the video if you want detailed info, if not, here’s the TLDR:

My shortlist as of September 2026

Qwen 3.8 27B — my “download tonight” all-rounder
Google Gemma 3 (4B / 12B / 27B) — start small, scale up
Mistral Magistral Small (24B) — Europe’s practical open option (and a bit of a surprise)
NVIDIA Nemotron Light / Lightning — if you already have an NVIDIA GPU
Meta Muse Glimmer (~30B distilled) — fallback when you have ~32GB+ RAM On a stronger Mac (often 64GB+), DwarfStar from antirez can push you into DeepSeek V4 / GLM 5.3 / Qwen territory that would otherwise want DGX Spark-class hardware.

General hardware requirements: you’d want a silicon Mac (Mac Book Pro or Mac Mini Studio) or an Nvidia based Intel machine (DGX Spark, Framework, etc). Memory rule of thumb: 16GB barely fits and it’s a basic start, 32GB you can actually do useful things, more is always better. You’ll almost always run a 4-bit / 3-bit / 2-bit quantized, not the full file that the labs released.

This whole thing is not only about speed, or tokens per second. Local means your prompts, chat history, and cache stay on your machine. You can run inference even without the internet. This time around, the ban on open source failed, but who know, it may succeed in the future. Knowing how to set up your own local AI is your insurance against this.

There’s also a full series about how to choose an open weights model in general. You may want to start here: How To Choose An Open Weights Model – Episode 1

And for more benefits: Running Local, Private AI Models, When and Why.