DEV Community: Dragos Roua

Can AI Improve Itself Without Training? Well, Maybe…

Dragos Roua — Thu, 05 Nov 2026 13:52:57 +0000

There’s something fascinating happening in the world of AI at this moment, and hardly anyone talks about it. AI can improve itself without training. Say that again? Yes! It turns out frontier models are capable of changing other, smaller models, by just editing their weights. Meaning they just look at the final file containing the model, that collection of matrices, they “understand” what’s there and they can take action, moving parts around, deleting them, editing them, etc.

That’s huge. It really is.

It all came from an experiment at MIT CSAIL, in which researchers took a large model (Astra) and pointed it at a much smaller one — Qwen 0.8B. They didn’t fine-tune anything. They just asked the big model to look inside the small model’s weights and see if anything could be improved.

Astra found a section of weights that was effectively blocking better math performance and removed it. After that single edit, the 0.8B model got substantially better at math. No training, no additional data, just a surgical change based on what the larger model could see in the weights.

They also ran a recognition test. Small models were trained on simple games (Pac-Man and others). The big model was given only the weights and asked to figure out what the small model was doing. It identified the correct game with over 70% probability. Again, no examples, no execution — just inspection of the weights.

Someone later tried a similar recognition task with the open-source Qwen 3.8 27B. It reached the same conclusion, though it took roughly 40 minutes instead of the faster result from the larger proprietary model.

What Self-Improving AI Can Realistically Do?

The researchers observed that this ability to usefully read and edit weights appeared “abruptly”. Nobody had a clear explanation for why it showed up when it did.

In the near term this is already interesting for practical work. If you have a fine-tuned small model that isn’t quite right, you can now hand it to a stronger model and ask it to look for problems or improvements without starting another training run. You can also get a first-pass audit of a small model’s likely behavior by having a larger one inspect the weights rather than running extensive tests.

Going forward, the open question is whether frontier models can eventually generate useful smaller models more directly — essentially producing the weights instead of only editing existing ones. If that becomes reliable, it removes a large chunk of the usual training compute from the process of making practical models.

That would affect how much large-scale GPU infrastructure is actually required for many use cases, and it would change the economics for the biggest labs. These are still speculations right now; the demonstrated result so far is inspection and targeted editing, not full model generation from scratch.

But the “abrupt” appearance of the capability hints at something bigger. Similar jumps have happened before with other abilities. If this one continues to develop, it could shift how we think about the cost and process of getting capable models into practical use.

I’ll keep an eye on the development of these experiments, and, if time and resources allow, I will try to reproduce.

If you like this kind of videos, like, share and subscribe to my YouTube channel!

Running A Billion Parameter Model (LFM3 1.7B) on a Raspberry Pi

Dragos Roua — Wed, 04 Nov 2026 16:34:30 +0000

As you already know, most of my local AI tests happen on a 16 GB MacBook Pro. Models in the 4B-9B range are the sweet spot there, and I’ve even run the Ternary Bonsai 2, at 27B (on a heavy quant). This time I wanted to see if you can run a more than one billion parameter model on much smaller hardware: a Raspberry Pi 4 with 4 GB of RAM.

I chose LFM3 1.7B from Liquid Foundation Models. It’s small enough that it should fit, but still large enough to be interesting. It’s the same model I tried in my previous test, as an in-browser model, with custom webGPU kernels. I added a small touchscreen on my Pi and ran everything locally — no SSH, nothing leaving the device.

Optimization Came from Parameter Tuning and a Custom Build of llama.cpp

Setup was straightforward but required a bit of tuning. I installed llama.cpp, played with the parameters a bit, mainly context size and thread count, then compiled a custom binary using ARM-specific flags for the Pi’s processor. The custom build increased generation speed from about 2 tokens per second to roughly 3 tokens per second. That’s a 50% improvement. At 3 tok/s it’s still slow by normal standards, but it’s close enough to reading speed that it almost starts to feel usable for light tasks. Almost.

I ran the same two tests I usually do. The text generation prompt (“what is a dolphin”) produced a surprisingly complete and detailed answer — one of the more thorough responses I’ve gotten from small models in these tests. The coding task was a simple word-count script in Python. The model generated working code and included basic instructions for running it. Both tests stayed entirely on the Pi. To my surprise, the coding tasks was finished quicker than text generation, but I suppose it was because the text about dolphins was pretty long, and just generating the tokens took some time.

A Raspberry Pi 4 with 4 GB is a low-end device. It’s not realistic to use it as a main coding machine. My medium term plan is to treat it as an orchestrator: it can handle medium or low-complexity tasks on its own and forward heavier work to more powerful machines. At 3 tokens per second it’s slow, but for that role it may be enough.

I’ll keep experimenting with further optimizations and different models on this setup. If the Pi can reliably run small agent-style tasks without needing to call out every time, it becomes a useful piece of a larger local stack.

If you like this kind of content, like, share and subscribe to my YouTube channel!

An AI Model in your Browser? LFM3 Omni 2.2B on a 16GB MBP

Dragos Roua — Tue, 03 Nov 2026 12:58:58 +0000

For the 10th episode of my series in which I test local AI models that fit on a 16GB MacBook Pro, I chose something different: a model that runs in your browser.

Most of the models I’ve tested so far need something like MLX Serve or a local runtime. Well, Liquid AI’s LFM3-Omni-2.2B runs entirely in the browser. You open a URL, the browser downloads roughly 1.6 GB the first time, and after that the model is just there.

It’s a 2.2B parameter omni model, and the “omni” means it can take text, images, or video and reply with text or speech.

Adjusting The Tests for the Model

I started with simple questions. “What is a dolphin?”, my standard one, came back cleanly, both as text and as speech. Then I skipped coding and started a conversation. “Who are you?” returned a short description of the model and what it can handle. It also answered questions about my Assess-Decide-Do framework with surprising clarity — very useful for a quick lookup.

A few practical notes. This is not a coding model. At 2.2B it is too small for anything that needs real tool use or multi-step code work. What it is good for is the low-friction daily assistant chores: summarizing a document, looking at a screenshot or short video, or answering a question out loud so it feels closer to talking to a junior assistant than to a chat window.

The voice output changes a lot. Once it is speaking back, the interaction is lighter than typing into a local server. On a 16 GB machine it sits comfortably because nothing else is running a model at the same time.

All in all, it is an almost surreal experience.

The browser port was done by Xenova using custom WebGPU kernels.

If you like this kind of content, like, share and subscribe to my YouTube channel!

[ Video ] Testing Ling 3.0 Tiny on my 16GB MacBook Pro

Dragos Roua — Mon, 02 Nov 2026 08:54:47 +0000

For the 9th episode of my series in which I test local AI models that fit on a 16GB MacBook Pro, I chose Ling 3.0 Tiny. It’s a 7.9 billion parameter mixture-of-experts model with 1.7 billion parameters active at any time. It comes from Inclusion AI (also referred to as Ant Group or Bai Ling), a lab based in Hangzhou and affiliated with Alibaba. I mention this because Alibaba is better known for the Qwen models, which I’ve covered in earlier tests in this series.

On a 16 GB MacBook Pro the model takes roughly 4–6 GB on disk. That leaves enough space for other applications, if you feel like, but I’d keep it tidy. It has a native 128K context window, but to conserve memory I kept the working context way lower — 32K.

The Standard Tests

I ran the same two checks I use for the rest of the series, through MLX Serve.

First, a plain text question (“what is a dolphin”). The model answered quickly and cleanly at 45 tokens per second. That is the highest generation speed I’ve measured so far, which puts Ling 3.0 Tiny on pole position.

Second, a small coding task: write a utility that counts words in files inside a folder. It requested tools twice — once to write the script, once to run it and verify — and produced a decent, working result without obvious bugs. Generation stayed just as fast during the whole process.

Overall it is a strong model for its size. Text generation is snappy and the coding output is usable. The only practical limit on 16 GB is context length. It works very well for short, clearly defined tasks. For longer or more open-ended work I would put a larger orchestrator model in front of it and only hand Ling well-specified pieces. Incidentally, this is the pattern I suggest across most of the small models I’ve tested on this hardware so far.

If you like this kind of content, like, share and subscribe to my YouTube channel!

[ Video ] Testing Apple Intelligence on my 16GB MacBook Pro

Dragos Roua — Sat, 31 Oct 2026 07:29:48 +0000

This is the 8th episode in the series where I’m testing a local AI model that runs on a 16GB MacBook Pro. Today I’m doing something special: I picked the model that Apple ships with Apple Intelligence (often called the Apple Foundation Model or AFM). It is already on your machine (Mac, iPhone, iPad) once you turn Apple Intelligence on. Apple does not distribute it the usual way (no Hugging Face weights, traditionally opaque), and there was no straightforward chat interface for it until recently.

I used an open-source tool apfel to get a normal chat interface:

https://github.com/Arthur-Ficial/apfel

It installs easily (Homebrew) and it provides a CLI and an OpenAI-compatible local server. No extra model download is needed.

I Ran The Two Standard Tests

First, the short question “What is a dolphin?” The answer appeared almost immediately. Across all the models I’ve run on this 16GB MacBook Pro, this was the fastest generation.

Second, the coding task: generate a small utility that counts words in files inside a folder. It defaulted to Python and produced a working script very quickly.

Because this is a rather unusual mode, a few technical notes. According to what I could find online, it is roughly a 3 billion parameter model. The space it occupies on disk is about 8 GB (Apple is opaque about the exact breakdown). Also, there is one caveat: some requests can be routed to Private Cloud Compute, so it is not always possible to know with high accuracy which ones stay fully on-device. On newer hardware (M3 and later) Apple can use a larger on-device variant that takes more space (up to around 12 GB).

Even the smaller model was impressive for speed and basic usability on a 16GB machine.

If you have other models that fit in 16GB and want them tested the same way, leave a comment. I read them.

And if you like this kind of content, like, share and subscribe to my YouTube channel!

Anthropic IPO — The Frightening Thing Nobody Talks About

Dragos Roua — Fri, 30 Oct 2026 09:46:21 +0000

Lately, I’ve been thinking a lot about the upcoming Anthropic IPO. There are 2 things that are out of proportion, and a third one that nobody talks about. I’ve made a small video about it (mostly ramblings).

If you prefer reading, stick around.

Anthropic released its IPO prospectus a few days ago. The actual process of going public looks essentially locked in, at a $2 trillion valuation. What follows are just my personal observations about how this might unfold. Not investment advice.

The Two Main Topics

First, the valuation. Two trillion dollars is roughly the GDP of Spain. On one side you have a real country with a functioning economy; on the other you have Anthropic. The comparison is crude, but it is useful for scale. The company does have real enterprise traction — Claude is being deployed on-premises and is generating meaningful corporate revenue — so a high valuation is not automatically absurd. Still, $2 trillion remains an extreme figure.

Second, the prospectus itself. It runs to about 400 pages, and a large part of it consists of the company warning that what it is building is extremely dangerous. This is consistent with the public messaging Anthropic has used for the past six to nine months: Dario Amodei and others have repeatedly highlighted AI risks while also suggesting that the company is particularly well positioned to mitigate them. A company that spends hundreds of pages telling potential investors “what we do is dangerous” is unusual. It may be the most alarming IPO document in recent memory.

The Hidden, Opaque Training Data

Third — and the part that nobody talks about — is what Anthropic already has inside. Public models (Opus, Sonnet, and the more restricted Fable and Mythos lines) are already very capable, we know it. Mythos-type models were trained on cybersecurity data and were deliberately held back or heavily gated because of risk. But internally the company almost certainly has stronger systems. On top of that, Anthropic has a large volume of real interaction data generated by people using its models, and it uses that data to keep training. Users pay for access and simultaneously improve the product.

That combination gives the company an unusually direct view of what people want and how they are likely to react. The $2 trillion number and the repeated danger warnings may not be pure signaling; they may rest on internal signals that outsiders cannot see.

But, if those signals turn out to be wrong, the size of the resulting correction would be historically large. A few hundred billion is one thing; two trillion is another.

The interesting question is not whether any individual can make money on the IPO. It is what happens to the broader environment if a valuation of this scale in the AI sector proves to be built on incorrect internal predictions.

We will find out soon enough. Until then we are in waiting mode.

[ Video ] Testing Spark-X2.6–4B on my 16GB MacBookPro

Dragos Roua — Thu, 29 Oct 2026 08:35:56 +0000

This is the seventh episode in my series of tests of a small model on a 16GB MacBook Pro. As usual the recording is raw — not sped up and not edited — so the timing and behavior are what you actually get.

Today we’re testing Spark-X2.6–4B (also written Spark X2.6 4B) from SparkLLM / XHToken. It is roughly 4.11 billion parameters, released under Apache 2.0. The architecture is the same as the previous Spark-X2.5–4B; the gains should come from improved post-training. The model card highlights better coding, agentic, and long-context performance and claims results that match or exceed some ~9B models on selected benchmarks (those are the team’s own numbers). It supports a 1 million token context window and three thinking levels (none, low, high). English and Chinese are the main languages. In this session I kept the context at 32k because of the memory limit on the MacBook.

I used the same two checks as always, running through MLX Serve.

First, the short question “What is a dolphin?” The answer was pretty good and complete. For a 4B model it ranks near the top of everything I’ve tested so far on this machine — top two or three.

Second, the coding task: generate a small utility that counts words in files inside a folder. The model asked for tools — first to write the file, then later to run or test it. It produced a bash script (Wordcount.sh) that works and is portable. Generation didn’t take especially long and the overall behavior was solid.

Taken together, Spark-X2.6–4B is a strong small model for text generation and light coding. It sits at roughly the same practical level as the Google Gemma and Nemotron Dory tests I did recently. Throughput is decent and you still have a bit of room left on a 16GB machine if you want to keep other small apps open.

If you have other models that fit comfortably in 16GB and you’d like them tested the same way, leave a comment. I read them. And if you like these tests, like and share and subscribe to my YouTube channel!

[ Video ] Testing Nemotron 3.5 Bolt E4B 4GB on my 16 GB MacBookPro

Dragos Roua — Wed, 28 Oct 2026 07:16:53 +0000

NVIDIA released Nemotron 3.5 Bolt E4B on October 23, 2026. It’s the smallest model in the new Bolt line — 8 billion total parameters with 4 billion effective, built by taking Google’s Gemma 4 E4B and running additional post-training on top of it. The focus is conversational agents, multi-turn tool calling, and lighter interactive or device-style use cases rather than heavy reasoning or pure coding. It supports a 64K context window, a thinking toggle, a conversational mode, and roughly 35 languages. NVIDIA also highlights that it produces around 80% less output tokens than similar models.

I tested the 4-bit version (available on Hugging Face) on my 16GB MacBook Pro, the same setup I use for other models in this series. On-disk size is between 4–6GB, depending on the exact quantization. The “4GB” in the name refers more to the target runtime memory envelope NVIDIA measured on certain hardware than to the download size itself.

The first check was a short open question: “What is a dolphin?” The model produced a solid, usable answer. It does some thinking before generating, and the tokens don’t appear instantly, but the total time stayed reasonable. For anything that doesn’t require maximum speed, it works reasonnably well.

The second check was a small coding task — generate a utility that counts words in files inside a folder. It chose to write a shell script rather than Python and delivered something that actually works. That was interesting; the model made a practical choice on its own.

Overall, Bolt E4B is usable on a 16GB MacBook Pro for day-to-day style work. It’s not the fastest model I’ve run in this memory class, but the answers are good enough and the token efficiency helps. Larger Bolt variants (the 74B class and above) exist for data-center or multi-machine home setups; those are outside what this series covers.

If you know other models that comfortably fit in 16GB and you’d like them tested the same way, leave a comment. If you like this kind of content, then like, share and subscribe to my YouTube channel!

[ Video ] Stories From The Vibe Coding Trenches – When the Agent (Claude) Deletes Your Files

Dragos Roua — Tue, 27 Oct 2026 11:53:30 +0000

In the third installment of the series I’m talking about something that is very likely to happen to you, especially if you’re newer to vibe coding.

The app that was vibecoded when this happened is Zen Tales. It’s a collection of Buddhist stories — koans, Jataka tales, pieces from the Buddha’s life — that I selected myself and carefully assembled. I’ve been exposed to Buddhism for a while and I believe it’s a good thing to have around. The app lets you listen to short stories (one to four minutes) and then reflect with a specially crafted AI model. It’s very simple, but, if we were to listen to the app store reviews, useful.

To get the audio I used text-to-speech. At the time Eleven Labs was quite hyped — but expensive, because the quality was high. During development I made small changes to some of the text files. Those changes required the corresponding audio files to be regenerated. I asked the agent (Claude, back then) to regenerate the files. There was no instruction to delete anything.

But it deleted ALL the recordings anyway. No backups. Then the regeneration from the new text didn’t go well. So I was left with work I had already paid for, gone.

What Happened Actually

The agent is not deterministic. It is stochastic. It samples from a distribution of possible next tokens. The same prompt can produce different outcomes on different runs. Something in the process went sideways — a misinterpreted step, an unexpected tool call, a context that drifted — and the files were deleted before new ones existed.

This is not rare. It is the normal behavior of the tools we are using. The model was not “trained to protect your files.” It was trained to be helpful and to complete tasks. Sometimes those two things collide in its “mind”.

The Recovery and the Takeaway

I got the files back because everything I build is in Git. I’ve been putting every project in a repo for more than twenty years. There was a commit. There was history. The probability of total loss dropped to near zero.

That is the practical lesson. Always have version control. Make copies. Make backups. If you are new to this and you are not yet comfortable with Git, spend a few days learning the basics before you start handing agents control of your files. The cost of that small investment is far lower than the cost of regenerating hours of work and burning token budget on something that already existed.

You can still move fast. You can still iterate. You just stop assuming the agent will do exactly what you expect every single time.

Because it won’t.

[ Video ] The User Knows Better Than You

Dragos Roua — Sat, 24 Oct 2026 06:55:12 +0000

In the first episode of this Stories from the Vibe Coding Trenches I talked about accidentally overpaying $150 in Cloudflare fees. The second episode is a bit more optimistic. Actually, it’s quite the opposite. It’s something that made me understand – for free – that the user knows better than you. And your mindset needs to adjust to this. It all happened while working on Shape Walk.

Shape Walk is just a simple walking app: you draw a shape (triangle, square, circle, heart, star…) on a map of your neighborhood, then you walk the outline. I wanted something that keeps me in shape, while feeling funny enough.

When I started building the app I had a few extra ideas. One of them was letting the app look around your current location and propose shapes for you. I didn’t implemented it. It felt complicated. I thought: “Users probably won’t use it.” Classic protective thinking.

After launch, a real person wrote back something close to this: “Instead of me trying to place a shape, why doesn’t the app just go out around me and find shapes?”

My first reaction was still the old one. Overly complicated. Then I opened my coding agent.

Half an hour later the feature was implemented. The version currently in the App Store can scan a couple of kilometers around you and suggest actual walkable squares or triangles.

The Real Story Isn’t the Feature, Though

The feature is nice and all. But the interesting part is what that revealed about my thinking patterns.

After 35 years of writing code I have well-worn neural pathways that push me toward the easy path and away from anything that feels hard. Those pathways were useful when every line had to be typed and every API integration carried real cost and risk. They are useless now. I would go even further and say they are harmful.

We can code almost anything we can clearly describe. The friction that used to force prioritization has dropped dramatically. When something still feels “too complicated” in my head, the odds are high that the complication is simply inexistent.

That doesn’t mean everything is free or that judgment disappeared. It means the default should shift from “this is probably too hard, skip it” to “let’s try it and see.” User requests that used to be polite “maybes” can now be fast experiments.

Key Takeaways

Don’t take features out just because they sound messy on paper.
Treat user feedback as cheap experiments, this is not scope creep.
Keep the coding agent prompts short and specific. My half-hour win happened because the request was clear.
Try to observe when old patterns are protecting you from work that no longer needs protecting.

Shape Walk still has the original draw-your-own-shape flow, and that’s the core part of it. The discovery mode is just an additional path for the days when you don’t feel like designing the route yourself. Both exist because the second one turned out to be easy once I stopped assuming it was hard.

If you’re coming from a traditional coding background, the biggest leverage right now might not be a better prompt or a better model. It might be retiring the part of your brain that still thinks some things are off-limits. There’s nothing off-limits now.

[ Video ] Running Nemotron Dory 1.0 4B on my 16 GB MacBookPro

Dragos Roua — Thu, 22 Oct 2026 14:46:56 +0000

This is the 5th episode in my series about local AI, in which I test models that can fit in a 16GB MacBook Pro. Today: Nemotron Dory 1.0, a 4B looped model from NVIDIA. The middle layers can run 1, 2, or 4 times at inference time, which means the model is slightly slower, but has a better output. I tested the 8-bit MLX quantized version, which amounts for around 4 GB, on my 16GB MacBook Pro.

Because the architecture is still very new, there is no native support in MLX Serve yet, so I had to test it it through the community dory-mlx package in the terminal. Surprisingly, that turned out to be useful, because you can watch the model’s internal reasoning in real time as it loops and revises.

On the memory side it behaved surprisingly well. Unlike Ternary Bonsai 2, which forced me to shut almost everything else down, Dory left enough headroom for other apps – didn’t use any, btw, but it was nice to see the space.

Speed wise, it wasn’t spectacular, though. A simple coding task (a word-counting utility that also had to write to disk) took around five minutes of back-and-forth thinking. The final code was sound, though the CLI interface cut the output short — which I believe to be a limitation of the current tooling rather than the model itself.

For pure text generation the results were also solid. The model stayed on topic, structured its answers, and didn’t overcomplicate things. I used my standard “what is a dolphin?” prompt.

All in all, Nemotron Dory 1.0 4B stays in a similar usability range to something like Gemma 4B, with the added advantage of being able to dial the amount of “thinking” up or down.

There’s an open pull request for proper Dory support in MLX-LM. Once that gets integrated, tool use and normal serving should become much smoother, and the output won’t be cut, like in my testing. Until then, the CLI route is usable if you’re patient and just want a small, competent local model that doesn’t eats out all your 16GB memory.

If you’re looking for a model that actually fits and runs on a base MacBook Pro without constant swapping, Dory is one of the more interesting recent additions.

[ Video ] Stories from the Vibe Coding Trenches – Accidentally Overpaid $150 to Cloudflare

Dragos Roua — Wed, 14 Oct 2026 12:55:49 +0000

I’m starting a new series of videos: stories from the vibe coding trenches, with real life situations encountered during my vibe coding endeavors. As you already know, I have 15 apps in the App Store already, and I started all this in February this year.

The first story is about how I overpaid $150 to Cloudflare, because of a rookie mistake. Full video is at the top of the article.

I launched Flight Lens because I wanted to monitor air disruptions and delays – I still travel a lot – plus a simple way to track ticket prices. I use a relatively expensive API for the airport / airborne planes data and the disruption engine is completely proprietary. It takes flight delay data, planes in the air, announced delays at takeoff and landing, and so on, and turns that into crystal clear, actionable information. Almost all the app work went into making that engine efficient enough to stay inside the paid API’s beginner plan.

The API itself was about $45 a month. The backend was deployed on Cloudflare for the usual $5. After almost 2 months of back and forth I launced and everything looked fine. People were using the app and the feedback was really nice.

Then the Cloudflare invoice arrived: $150.

I’ll do the math for your: that is three times the API cost. I literally had no idea what hit me. But then I drilled down the Cloudflare’s billing dashboard, and found out that database reads and database writes are separate units. While I was heavily optimizing the API side of the disruption engine, I had written roughly 150 million times in a month. I was nowhere near the read limits, but the writes had gone way over.

So, I went back in and optimized again, this time watching both the API limits and the database write ceiling. I also set up billing alerts so I get notified before anything climbs too far (in all fairness, those alerts should have been there from the start).

Even with more than thirty years of writing code, these newer services still surprise me – especially if skip how they actually charge. The takeaway here is “just pay attention”, nobody will warn you if you ht some obscure paying ceiling.

The next stories in this series will be lighter, this one was the only expensive one (so far).

If you like this kind of content, make sure to like, share and subscribe to my YouTube channel.