How a 270M-parameter model on your phone turns a dictated note into contacts

Valentin Telegin — Thu, 22 Oct 2026 01:34:19 +0000

Circles is a birthday reminder built around people’s circles: a friend, their partner, their kids, their parents. The slowest part of any app like this is typing them in. Adding one family means a form per person — name, relation, gender, date. Saying it is faster:

“Dave has a wife Jen, she turns 40 on March 3. Their son Leo is 6.”

That note should become three people: Dave, his wife Jen (birthday March 3, turning 40) and their son Leo (about 6). This is how we made that work on the phone — in English, Spanish and Russian — without sending anyone’s family to a server.

Why not a cloud model

A cloud LLM would parse the note in a couple of seconds. But what people dictate here is exactly what they are most careful with: names of children, ages, who is married to whom. Circles has no accounts and no servers, and we wanted to keep it that way. On-device also means it works offline and costs nothing per note.

First try: a general small model with a long prompt

We started with Gemma 4 E2B: an instruction, four worked examples and a JSON schema for constrained decoding. Quality was decent — 95% of fields right on a 50-note test set — but on a Pixel 8 Pro a note took about 11 seconds on the GPU and 21 on the CPU. Reading the ~1,100-token prompt was the cheap part (~1.2 s on the GPU). Generating the answer at ~10 tokens a second was the problem.

Smaller off-the-shelf models did not help. Some community builds answered with an empty string even to “What is the capital of Spain?”; another produced broken JSON in more than half of the notes and looped until the token limit.

What worked: a tiny model taught one job

Instead of explaining the task to a general model in every prompt, we taught a tiny model the task itself:

Model: Gemma 3 270M, fully fine-tuned.
Data: a generator of synthetic notes with known answers — 18,000 notes in English, Spanish and Russian.
Training: MLX on a MacBook Air (M5, 16 GB), 3,400 steps, about 40 minutes.
Prompt: just the note — no instruction, no examples. The answer is compact JSON without empty fields.
Runtime: LiteRT-LM, int8, a 296 MB file, on the CPU.

On the same Pixel 8 Pro that is about 1 second per note (90th percentile 1.2 s), ~46 tokens a second. On a test set written after training, it was also more accurate than the model ten times its size with the long prompt. Running on the CPU turned out to be a feature: it does not need a flagship’s GPU or NPU.

Gemma 4 E2B + instruction and examples: 17 of 30 held-out notes fully right, 10.7 s per note (GPU)

Gemma 3 270M, fine-tuned: 21 of 30 held-out notes fully right, 1.0 s per note (CPU)

The data generator is the product

Every improvement came from the generator, not from the training code. The loop: write a fresh test set the model has never influenced, look at what it gets wrong, teach the generator that pattern, retrain, and check on the next fresh set. Some of what it taught us:

Notes have more than one sentence. An early model found Jen but lost Leo: “Their son…” in a second sentence never appeared in training. Adding two- and three-sentence notes took a multi-sentence test from 5 to 17 of 24 notes fully right.
Speech is not text. Real dictation has no commas, writes numbers as digits or words (“October twenty second”, “mil novecientos cuarenta y ocho”), and has fillers and corrections (“the seventh — no, the eighth”). Rewriting a third of the notes that way, then training on what phone dictation actually produces, took real dictated notes from 6 to 13 of 15.
Some languages have grammar. In Russian, names change with case: “у Димы жена Лена” (Dima’s wife Lena) must give “Dima”, not “Dimy”. The generator inflects names, patronymics and nicknames.
People talk about the owner, too. “Vladlen is Alexandra’s husband, he’s 43, she’s 44” — the model only returned peoplearound someone and dropped the owner’s own age. A "self" marker in the answer fixed it.
Continue, don’t restart. Training from scratch on the new mix learned the new trick but got worse at ordinary notes. Continuing from the previous model with a small learning rate kept both. The current model gets 14 of 15 real dictated notes and 24 of 24 “spoken” test notes fully right.

The app checks the model

The model only extracts; the app does the rest. It works out birth years from ages, drops an age the note never said, matches names to people already in the circle, merges duplicates and turns relations into “son”, “hija” or “дочь” with the right grammar. Nothing is saved until you review the cards. Three of four notes need no edits; the rest usually need one field.

Shipping a 296 MB model

The model is not in the app binary. On Android it is an on-demand Play Asset Delivery pack; on iPhone, an Apple-hosted Background Assets pack (iOS 26.4 and later). It downloads the first time you open dictation, so people who never use it never download it.

Try it

Circles is free for up to 10 circles; dictation is part of Pro. Questions about the training setup or LiteRT-LM on iOS and Android: support@circlesbirthdays.com.

Circles is on Google Play: https://play.google.com/store/apps/details?id=com.circlesbirthdays.app

Circles is on AppStore: https://apps.apple.com/app/id6824990348

Website: https://circlesbirthdays.com

Originally published at https://circlesbirthdays.com on October 21, 2026.

DEV Community: Valentin Telegin