For an ordinary voice product, the choice between processing on the device and processing in the cloud is a question of money and latency. For a children's product it is a question of market access, and it gets settled before the first line of code. We are building WhyWave, a voice companion for children aged three to five, and we went looking for how much of the pipeline actually fits on the device. What we found along the way was an unpleasant symmetry: the smaller the model, the worse it understands children specifically, and the gap here is wider than it is on adult speech.
On why the cloud is closed to a children's product
Rule 1.3 of the App Store Review Guidelines forbids apps in the Kids Category from sending personally identifiable information or device information to third parties. A child's voice is exactly that: the definition in 16 CFR § 312.2 includes an audio file containing a child's voice outright, with no further condition such as a name or an identifier attached to it.
The usual voice pipeline goes microphone, cloud recognition, cloud language model, cloud synthesis. Every arrow there hands a child's voice to an outside service. On paper this collides with rule 1.3 head on, and while Apple softens the wording on its own developer page with a caveat about explicit parental consent, the contradiction between its two texts has never been resolved.
Then the model vendors. The Gemini API Additional Terms and section 20.d of the Google Cloud Service Specific Terms prohibit using Google's generative services inside a product directed at people under eighteen. OpenAI and Anthropic took the other route and published requirements for serving minors, though those requirements include zero data retention, age verification and audit rights.
Local processing changes the legal picture entirely, and that is the main argument in its favour. If the audio never leaves the device, the operator never receives it, so no collection occurs in the COPPA sense. There is nothing for rule 1.3 to prohibit, because no third party is in the diagram. Vendor terms do not apply, because there is no vendor either.
On what already fits on the device
Over the last two years the local pipeline stopped being exotic, and the numbers make that case better than any argument.
Recognition first. The WhisperKit team published work where Whisper Large v3 Turbo, a billion parameters, runs on the Neural Engine and is compared with cloud systems directly: gpt-4o-transcribe, Deepgram nova-3, Fireworks large-v3-turbo. The result is 0.46 seconds of latency at 2.2 per cent word error, which means the local system matched the fastest cloud system and beat all of them on accuracy. That was measured on a MacBook Pro with an M3 Max chip, though, and the caveat is worth keeping in mind. Separately they show that self-distilling the encoder with block-causal masks cuts encoder latency by 65 per cent, from 602 to 218 milliseconds, while accuracy stays within one per cent of the original.
Further down in weight sits Moonshine. Tiny is 27.1 million parameters, Base is 61.5 million. Dropping the mandatory zero-padding to thirty seconds that Whisper requires gives a fivefold reduction in compute on a ten-second segment, at an average word error of 12.66 against 12.81 for Whisper tiny.en. The authors say plainly what this is for: on cheap ARM processors Whisper tiny.en had a firm lower latency bound of around five hundred milliseconds, and a variable-length input removes it.
Synthesis. Kokoro is 82 million parameters, Apache-licensed weights, roughly 327 megabytes, 54 voices across eight languages. For scale, that is less than the average mobile game.
The language model. Apple Intelligence is a three-billion-parameter model running on the Neural Engine in a phone you can buy today. Models of that class on the device are ordinary now.
It all adds up to a sense that the problem is solved. At WhyWave that sense lasted exactly until we looked at how any of it works on children.
On the price the child pays
There is a table in the children's ASR benchmark worth reading slowly. These are Whisper-family models run without fine-tuning on the OGI Kids corpus: tiny at 39 million parameters returns 53.8 per cent word error, base at 74 million returns 38.0, small 25.4, medium 20.8, large-v3 at one and a half billion 19.9.
From the smallest model to the largest, the error falls by nearly a factor of three. On spontaneous children's speech in the MyST corpus the same row runs from 20.6 at tiny to 12.5 at large, a drop of less than half. The harder the material, the more every parameter you cut away costs you.
Now set that against the previous section. What lives on a device without effort is precisely tiny and base. By this table, those are the models that get every second word wrong when a child reads a word list aloud. Local processing, which solves the entire legal problem in one move, is bought at the price of accuracy on exactly the audience the whole thing was for.
One caveat, without which the picture flatters itself. MyST is third to fifth graders, OGI Kids runs from kindergarten upward, about fifty hours from 1,100 speakers. Preschoolers are either absent from these corpora or lost inside them. Our three to five year olds are territory with no public measurements at all, so 53.8 per cent at tiny should be read as an optimistic ceiling and not a worst case.
On what to do about it
The same paper has a second table that turns all of this around. After fine-tuning on children's speech, Whisper-tiny returns 2.7 per cent word error on OGI instead of 40.1 without it. On MyST it gets 11.6 instead of 20.6. Base drops to 2.0 after fine-tuning, medium to 1.6, so the gap between the tiny model and the large one nearly disappears. Fine-tuning tiny took two hours on two GPUs.
The conclusion runs opposite to what the previous section was building towards. What the model was trained on weighs more than how large it is. A 39-million-parameter model that has seen children's speech beats a model forty times its size that has not. For a product that needs to compute locally, this is the best news available: the thing worth fine-tuning is exactly the thing that fits on the phone.
All of which runs into data, of course. There is no public transcribed corpus of preschool speech, and collecting one yourself means recording three-year-olds, which starts up precisely the machinery of consent, storage and audit that local processing was supposed to spare you. WhyWave is still looking for the way out of that circle, and if you have one, write to us.
If you are building a voice product for adults, one thought from all this is useful to you. Numbers like "2.2 per cent word error" were obtained on speech that resembles the speech the models were trained on. Worth asking in advance how much your users resemble that sample, and worth looking for the section of the documentation that describes the training data. You probably will not find one.
Top comments (0)