The Small Model Was the Stock Answer

"What should we do today?" came back as "go to the market." The words were generated, and they still sounded like a line that had been saved in advance.

Savindu Thathsara, savithathsara.me. This is a Hearthlen note from the laptop.

The voice model was SmolLM2 at 360 million parameters. A model that size can finish a short sentence. It cannot hold the detail you just said and do something with it, so the sentence comes out generic. The voice agent now keeps Qwen2.5-3B-Instruct loaded, 4-bit, 2,104,932,768 bytes. Asked "I have two hours free and it is raining. What should I do?" it answered: "Since it's raining and you have two hours free, why not take this opportunity to read a book?" That was 39 tokens in 3017 ms. It used the rain and the two hours. The smaller model never did.

The face shows Thinking while those tokens are generated. A plain command still goes to the desk. The conversation this replaces is in the reply comes from the model.

---

Savindu Thathsara builds practical AI and automation in London. This series is the public log of an offline AI operating system he is writing and learning at the same time.

• Site: https://savithathsara.me
• Email: [e-mail]
• DEV: https://dev.to/savinduthathsara