Building the on-device ML pipeline that powers phone-based sleep analysis

Most sleep-detection demos run a model server-side on uploaded audio. That is the easy path and also the wrong one: it asks users to ship their bedroom audio to a cloud bucket, and it makes the product a privacy liability every time you explain it. The harder and right path is a full CoreML pipeline that takes microphone samples to snore events on-device, in the dark, with no network. Here is how SleepTrace actually wires that together and why each choice matters for overnight battery and user trust.

The pipeline stages, end to end

1. Capture as a 12.8 kHz mono ring buffer via AVAudioInputNode (pull mode), with a background-task assertion renewed on a 25-second cadence so iOS does not reclaim the budget. Full architectural notes in the field writeup here.
2. Window into 2-second frames with 50% overlap. Each frame is a fixed cost; batching them in and out of the queue smooths CPU usage.
3. Feature extraction is a 40-coefficient mel filterbank plus spectral tilt, zero-crossing density, and harmonic-to-noise ratio — the exact discriminators documented on the blog.
4. Score each frame through a CoreML model exported from a PyTorch training job, quantized to 16-bit for size.
5. Smooth the frame-level posteriors into event boundaries (snore start/end, breathing pause) in a lightweight temporal pass — no full-night attention model on the phone.
6. Emit events to a local log; upload, if opted in, is aggregate counts only.

Why on-device is the whole point

Moving the scoring to CoreML is not just a privacy posture. It changes the resource contract:

• Latency. A frame is classified in under 2 ms on recent silicon; the OS never has to decide whether to wake a co-provisioned network call.
• Battery. There is no radio cost. Radio wakeups dominate overnight phone battery drain, and audio-classification workloads run entirely offline.
• Honesty. You cannot accidentally leak the raw audio if the raw audio never reaches a host boundary. The trust model is baked in by construction.

Quantization and the size budget

The full unquantized model is ~80 MB. Quantizing to 16-bit int8 with post-training quantization lands it under 20 MB and changes per-frame accuracy by under 1%. That 20 MB is the ceiling we budget against because iOS does not like resident CoreML models in the multi-hundred-megabyte range on a phone sharing memory with the OS all night.

The one place the model is not the bottleneck

Feature engineering is. The mel filterbank and the spectral-tilt coefficient carry most of the discriminative signal between snoring and speech, and they are trivial to compute. A well-featurized linear separator often matches a much larger neural net here precisely because the feature space is doing the work. Ship the features first; ship the bigger model only if the features alone miss your false-positive budget.

This is the architecture that lets a $600 phone replace a $300 wearable on the nightstand — and it only became feasible once every stage moved on-device.