DEV Community

Valentin Telegin
Valentin Telegin

Posted on Edited on Originally published at circlesbirthdays.com

How a 270M-parameter model on your phone turns a dictated note into contacts

Circles is a birthday reminder built around people’s circles: a friend, their partner, their kids, their parents. The slowest part of any app like this is typing them in. Adding one family means a form per person — name, relation, gender, date. Saying it is faster:

“Dave has a wife Jen, she turns 40 on March 3. Their son Leo is 6.”

That note should become three people: Dave, his wife Jen (birthday March 3, turning 40) and their son Leo (about 6). This is how we made that work on the phone — in English, Spanish and Russian — without sending anyone’s family to a server.

Why not a cloud model

A cloud LLM would parse the note in a couple of seconds. But what people dictate here is exactly what they are most careful with: names of children, ages, who is married to whom. Circles has no accounts and no servers, and we wanted to keep it that way. On-device also means it works offline and costs nothing per note.

First try: a general small model with a long prompt

We started with Gemma 4 E2B: an instruction, four worked examples and a JSON schema for constrained decoding. Quality was decent — 95% of fields right on a 50-note test set — but on a Pixel 8 Pro a note took about 11 seconds on the GPU and 21 on the CPU. Reading the ~1,100-token prompt was the cheap part (~1.2 s on the GPU). Generating the answer at ~10 tokens a second was the problem.

Smaller off-the-shelf models did not help. Some community builds answered with an empty string even to “What is the capital of Spain?”; another produced broken JSON in more than half of the notes and looped until the token limit.

What worked: a tiny model taught one job

Instead of explaining the task to a general model in every prompt, we taught a tiny model the task itself:

  • Model: Gemma 3 270M, fully fine-tuned.
  • Data: a generator of synthetic notes with known answers — 18,000 notes in English, Spanish and Russian.
  • Training: MLX on a MacBook Air (M5, 16 GB), 3,400 steps, about 40 minutes.
  • Prompt: just the note — no instruction, no examples. The answer is compact JSON without empty fields.
  • Runtime: LiteRT-LM, int8, a 296 MB file, on the CPU.

On the same Pixel 8 Pro that is about 1 second per note (90th percentile 1.2 s), ~46 tokens a second. On a test set written after training, it was also more accurate than the model ten times its size with the long prompt. Running on the CPU turned out to be a feature: it does not need a flagship’s GPU or NPU.

Gemma 4 E2B + instruction and examples: 17 of 30 held-out notes fully right, 10.7 s per note (GPU)

Gemma 3 270M, fine-tuned: 21 of 30 held-out notes fully right, 1.0 s per note (CPU)

The data generator is the product

Every improvement came from the generator, not from the training code. The loop: write a fresh test set the model has never influenced, look at what it gets wrong, teach the generator that pattern, retrain, and check on the next fresh set. Some of what it taught us:

  • Notes have more than one sentence. An early model found Jen but lost Leo: “Their son…” in a second sentence never appeared in training. Adding two- and three-sentence notes took a multi-sentence test from 5 to 17 of 24 notes fully right.
  • Speech is not text. Real dictation has no commas, writes numbers as digits or words (“October twenty second”, “mil novecientos cuarenta y ocho”), and has fillers and corrections (“the seventh — no, the eighth”). Rewriting a third of the notes that way, then training on what phone dictation actually produces, took real dictated notes from 6 to 13 of 15.
  • Some languages have grammar. In Russian, names change with case: “у Димы жена Лена” (Dima’s wife Lena) must give “Dima”, not “Dimy”. The generator inflects names, patronymics and nicknames.
  • People talk about the owner, too. “Vladlen is Alexandra’s husband, he’s 43, she’s 44” — the model only returned peoplearound someone and dropped the owner’s own age. A "self" marker in the answer fixed it.
  • Continue, don’t restart. Training from scratch on the new mix learned the new trick but got worse at ordinary notes. Continuing from the previous model with a small learning rate kept both. The current model gets 14 of 15 real dictated notes and 24 of 24 “spoken” test notes fully right.

The app checks the model

The model only extracts; the app does the rest. It works out birth years from ages, drops an age the note never said, matches names to people already in the circle, merges duplicates and turns relations into “son”, “hija” or “дочь” with the right grammar. Nothing is saved until you review the cards. Three of four notes need no edits; the rest usually need one field.

Shipping a 296 MB model

The model is not in the app binary. On Android it is an on-demand Play Asset Delivery pack; on iPhone, an Apple-hosted Background Assets pack (iOS 26.4 and later). It downloads the first time you open dictation, so people who never use it never download it.

Try it

Circles is free for up to 10 circles; dictation is part of Pro. Questions about the training setup or LiteRT-LM on iOS and Android: support@circlesbirthdays.com.

Circles is on Google Play: https://play.google.com/store/apps/details?id=com.circlesbirthdays.app

Circles is on AppStore: https://apps.apple.com/app/id6824990348

Website: https://circlesbirthdays.com

Originally published at https://circlesbirthdays.com on October 21, 2026.

Top comments (5)

Collapse
 
oddeggs profile image
Odd Eggs Studio •

Speech is not text, and the "the seventh, no, the eighth" corrections are so real. Our spoken-spend parser is plain rules rather than a model, and simply taking the last number said handles the "forty, no, forty-five k" case for free.

Collapse
 
valentin_telegin profile image
Valentin Telegin •

Thanks! "Last number wins" is a neat trick, and for a single amount it's hard to beat. Our notes usually have several values in one breath: "Leo is 6, and his sister Mia turns 9 on the seventh, no, the eighth." There the correction has to replace the right date, not just the last number, and that's where rules got messy for us. So we rewrote a third of the training notes into speech-like text with fillers and corrections, and that did more for real dictation than anything else we tried.

Do you ever see corrections that aren't numbers, like "with Sarah, sorry, Sara"? Or is the amount the only thing people correct?

Collapse
 
oddeggs profile image
Odd Eggs Studio •

Mostly the amount, since a spend is really just a name and a number. When someone says "phở, no, bún" both words land in the name chip and they fix it with one tap before saving, so we never tried parsing it, and your one-breath multi-value notes sound like a much harder problem.

Thread Thread
 
valentin_telegin profile image
Valentin Telegin •

That makes sense: with one name and one number, a quick tap to fix is cheaper than any parser. We ended up in a similar place. The model does its best with corrections, but nothing is saved until you check the cards, and most notes need no edits or just one field. The review step catches what parsing can't. Good luck with the spend parser!

Thread Thread
 
oddeggs profile image
Odd Eggs Studio •

Same rule here: nothing is kept until you've seen the card, so the parser only has to be good, not perfect. Good luck with the contacts model!