Not long ago I didn't know what the word deploy meant. Now there's a model running on my laptop that I trained myself. Here's how it went, where I got things wrong, and how good-looking numbers almost fooled me.
Why
I have a personal AI assistant called Pulsara. It runs on external models, and at some point I asked myself what happens if those models get shut off, get more expensive, or just get worse. The assistant stops working and there's nothing I can do about it.
So I wanted a model of my own. It lives on my computer, and nobody can switch it off from outside.
The second goal was more interesting. I didn't want the model to know what's inside books. I wanted it to know how to read. Adler and Van Doren's How to Read a Book describes a method: first figure out what kind of text it is and what it says in one sentence, then break down its parts, terms and arguments, and only after that argue with it. Knowing a book means you can retell it. Owning the method means you can work through a text you've never seen. I wanted the second one.
The setup
A MacBook Air M5 with 16 GB of memory. No server, no GPUs.
The biggest thing that fits is Qwen3 with 8 billion parameters in compressed form, 4.4 GB. Anything larger won't fit, because training needs memory too. I didn't retrain the whole model. I trained a small add-on on top of it (the technique is called LoRA). The base model stays frozen and only a set of small extra matrices changes.
My first day wasn't spent training. It was spent getting anything to install at all. My Mac's terminal turned out to be running in compatibility mode for old Intel chips, and the library I needed simply wouldn't install there. Tutorials don't mention things like that.
The data
I put together 576 training examples. About 150 on the method itself, 160 walkthroughs of texts in different genres, 120 on behavior (for example, what to say when asked whether it has read a book), and 110 ordinary conversations with no Adler at all.
That last group mattered. Without it, the model started applying the method to a request for coffee brewing tips.
Four attempts
Version one was cautious, and the result was weak: 13 out of 35 method markers on my test questions. The model recognized Adler's name but analyzed texts its own way.
In version two I changed two settings at once. Training collapsed halfway through, and I never found out which change killed it. Since then I change one thing at a time.
Version three scored 27 out of 35. The method started carrying over to unfamiliar texts. But the model confidently claimed to have read books it had never been shown, and passed judgment on things it hadn't seen.
For version four I didn't touch the settings at all. I only added 74 examples on holding back judgment. It scored 33 out of 35, and the false claims about reading went away. At that stage the data mattered more than the parameters.
Where the numbers lied
33 out of 35 looked like a win. Then I sat down and read the answers myself, on real books the model had never seen: Eric Berne and a recent issue of Harvard Business Review.
It was a different picture. The model summarized the main ideas fairly well, but it added details that weren't in the text. About an experiment in one article, it wrote that half the participants worked with AI and half without. The article said nothing of the kind. It also promised to come back with a report in a week, even though it has no memory between conversations.
My automated check looked for signs of the method in each answer: did it name the type of text, did it state the main idea in one sentence. It could see the form. It never checked whether the content was true.
There were worse traps. The server that serves the model was silently dropping my add-on and answering with the untrained model. No error, no warning, and the answers looked fine at a glance. I only caught it because I compared the server's answers word for word against saved ones. Later I found that the model with the add-on merged into its weights answers differently from the same model with the add-on kept separate. Merging compresses the weights again, and part of the training gets lost.
What I learned
Training a model on a laptop is doable today, and it isn't the hard part. The hard part is finding out what it actually learned.
A metric you keep adjusting starts showing you what you want to see. So now I write the exam questions before the model answers them and keep them separate from the training material.
A small model picks up manner and working order well. It doesn't get smarter: 8 billion parameters after training are still 8 billion.
About me: I wrote the code together with AI assistants. I couldn't write it at that level on my own. My part was deciding what to test, not trusting pleasant numbers, and reading the answers myself.
What's next
The next version is about accuracy: grounding answers in quotes from the text and not adding things that aren't there. After that I'll connect it to Pulsara.
If you're building something of your own with AI without an engineering background, tell me where you got stuck. I'd like to compare notes.
Originally published at ilinmaks.com.
Top comments (0)