Hello, everyone.
If meeting and note transcription can run entirely on a Mac, the audio never has to be sent to an external service. But how accurate and responsive is that experience while someone is speaking Japanese?
Today, I am feeding Japanese audio incrementally into Apple's SpeechAnalyzer and measuring recognition accuracy and partial-result latency. I also built a local browser tool for trying it with a microphone.
The short answer is that both configurations achieved a 0% character error rate (CER) in all three trials on a 14.171-second synthetic conversation. In progressive mode, however, the first text arrived after about 1.09 seconds, and partial-result delivery latency was about 1.01 seconds at p95. The content was accurate, but the text did not appear immediately after each word was spoken.
What Is SpeechAnalyzer?
SpeechAnalyzer is Apple's new speech-to-text API, announced at WWDC25 in June 2025 and introduced with the iOS 26 and macOS 26 generation.
Its SpeechTranscriber module uses a new general-purpose conversational speech-recognition model. Apple describes it as faster and more flexible than the previous model and suitable for long-form or distant audio such as meetings and lectures. Processing runs on the device.
This is not an open-source model whose weights can be downloaded and redistributed independently. AssetInventory downloads the required model from Apple's servers, while the operating system stores and updates it. Apple does not present a standalone open-source license for the model; use of the Speech framework and its model is subject to the applicable Apple OS, Xcode, SDK, and developer terms. The lab code created for this experiment uses the MIT License.
Roles and Data Flow
This experiment does not combine multiple recognition models. It uses one ja_JP SpeechTranscriber model. The surrounding path is:
browser microphone (normally 48 kHz Float32)
-> AudioWorklet
-> WebSocket (localhost)
-> Python bridge (resample to 16 kHz)
-> Swift CLI (convert to Int16 PCM)
-> SpeechAnalyzer + SpeechTranscriber
-> partial / final results
-> browser display
PCM is a basic audio representation that stores the sound wave as a sequence of numbers. The Python bridge and Swift CLI only prepare the format; they do not recognize speech. Audio remains within localhost and is neither uploaded nor saved.
What I Tested
The experiment asks three questions:
- Can it transcribe a known Japanese conversation correctly?
- How long do the first and subsequent partial results take?
- Can a browser microphone be connected to the on-device SpeechAnalyzer path?
The complete code and JSON measurement report are available in the apple-speech-analyzer-streaming-asr lab in kiarina/labs.
Reproducing the Lab
You will need an Apple Silicon Mac, macOS 26 or later, Xcode Command Line Tools, mise, uv, and FFmpeg. These commands download the shared audio asset, run the tests, build a release binary, and execute three trials per configuration.
git clone --depth 1 --filter=blob:none --sparse \
https://github.com/kiarina/labs.git
cd labs
git sparse-checkout set .gitignore .mise/tasks Makefile mise.toml \
2026/07/20/apple-speech-analyzer-streaming-asr
mise -C 2026/07/20/apple-speech-analyzer-streaming-asr run
To try a live microphone in the browser, run:
mise -C 2026/07/20/apple-speech-analyzer-streaming-asr run demo
Partial text appears in lime and finalized text in white. The local server has no authentication, so keep it on 127.0.0.1 and do not expose it publicly.
Test Conditions
I fed a 14.171-second synthetic two-speaker conversation in 100 ms chunks, paced to match real time. I compared two presets:
-
transcription: prioritizes accuracy and returns finalized results -
progressiveTranscription: also returns the accumulated in-progress transcript
The main environment was a MacBook Pro with an Apple M1 Max and 64 GB of memory, macOS 26.5.2, and Swift 6.3.3. The locale was ja_JP, and input was 16 kHz mono Int16 PCM.
I measured differences from the reference with CER. It is the proportion of character substitutions, deletions, and insertions needed to match the correct text. This experiment ignored whitespace and punctuation during that comparison.
Results
| Preset | CER | Partial results | First partial | Partial delivery p50 / p95 | Final delivery |
|---|---|---|---|---|---|
| transcription | 0.0% × 3 | 0 | — | — | 1.160–1.215 s |
| progressive | 0.0% × 3 | 93, 93, 94 | 1.082–1.101 s | 0.526 / 1.014 s | 1.033–1.223 s |
End-to-end real-time factor (RTF), from input start to finalization, was 1.072–1.086. An RTF of 1.0 means the full processing time equals the audio duration. Because this test intentionally waits while feeding audio at real-time speed, that figure is not the model's isolated compute speed.
All Six Utterances Retained Their Content
After ignoring punctuation and whitespace, all six utterances matched the reference in every trial. The phrase containing the number, あと5分くらいかな (“about five more minutes”), was also preserved.
Punctuation varied between trials and presets. For example, one result joined そっちはこっちは, while another inserted a full stop between the two phrases. A 0% CER here does not mean that every output was equally readable.
Progressive Display Lagged by About One Second
progressiveTranscription returned its first partial after about 1.09 seconds and then kept updating the accumulated text. Across all 280 partial results, delivery lag relative to the corresponding audio time was 0.526 seconds at p50 and 1.014 seconds at p95.
p95 means that 95% of the results arrived within that amount of time. The preregistered targets—under one second for both the first partial and p95 delivery—were narrowly missed.
The Live-Microphone Path Also Worked
The automated end-to-end test sent 48 kHz audio in browser-sized 128-frame chunks, resampled it to 16 kHz, and passed it to SpeechAnalyzer. It received 93 partial results and one final result, with the same final content as the benchmark.
In an informal Chrome trial with a MacBook Pro's built-in microphone, one user also perceived about one second of display delay. Recognition errors seemed infrequent, but 十分 (“enough”) was rendered as 10分 (“ten minutes”), and some punctuation was missing. This was a subjective single-person check, not part of the measured CER evaluation.
A Plain-Language Reading of the Results
- It captured the spoken content accurately in one short, clean conversation.
This was only one synthetic recording. A noisy room or unfamiliar proper nouns may produce very different results.
- It can update text while someone speaks, but the display is roughly one second behind.
That may be acceptable for meeting notes or draft captions, but it may feel slow for a voice command that should react immediately after one word.
- Speech can be processed without sending it away from the device.
The model runs on the Mac. Its initial asset download and later updates still come from Apple's servers.
Limitations and Implementation Notes
- The evaluation covered one clean, 14.171-second synthetic conversation and three trials per preset.
- It did not score punctuation quality.
- It did not test noise, reverberation, dialects, proper nouns, or long streams.
- The exact version and hash of Apple's system-managed model cannot be pinned.
- The benchmark excludes microphone buffering and browser permission time.
The implementation had to match the model's PCM type as well as its sample rate. Passing Float32 directly failed, while conversion to the 16 kHz Int16 format requested by SpeechAnalyzer worked. Reproductions should record the OS build, Speech framework, and locale together.
My Takeaway
Even though this was only one short conversation, a 0% CER across every on-device trial was better than I expected. The roughly one-second display delay was visible, but it did not feel extremely slow during informal free speech.
Punctuation was less stable than content recognition, so a production tool would benefit from a readability cleanup stage. The current path looks useful for local meeting notes or draft captions when audio must not leave the device.
Top comments (0)