Azure Speech will give a spoken word one number, PronScore. For a Chinese speaker learning American English, that number is the wrong thing to teach with. Mandarin does not have /θ/, /ð/, /æ/, or the American flap. The word can score in the 80s while one of those phonemes is the actual failure.
I built Vocalace around that. The screen shows one word. The pass/fail decision and the red mark are two different scores, on purpose.
What the assessment payload actually contains
A successful response has RecognitionStatus: "Success" and an NBest array. I only read NBest[0]. The word score is PronScore, and I floor it. If the field is missing, the attempt scores 0. I do not average the alternatives.
Inside that first hypothesis, each word has a Phonemes array. Each phoneme has its own AccuracyScore. That list is what I can mark. PronScore is what I can use to decide whether the attempt counts.
The JSON is treated as unknown and parsed field by field. A missing Words array, a phoneme with no AccuracyScore, or a non-numeric score is skipped. One bad node does not throw away the rest of the attempt.
Two thresholds
A phoneme is marked when its AccuracyScore is under 60. The attempt passes only when the floored PronScore is at least 90.
These are not two views of the same cutoff. A word can pass 90 and still contain a phoneme under 60, because PronScore is a whole-word score. A word can also fail 90 with no marked phoneme, when several phonemes are merely mediocre. Marking every phoneme below 90 would light up the whole transcription. Using 90 as the mark threshold and the pass threshold would make those two failures look the same.
So the UI shows the sounds that were clearly off. The scheduler, not the red marks, decides whether the word is learned.
The attempt does not master a word
Passing once is not mastery. The states are unseen, confirming, and mastered.
- unseen and a score of at least 90 moves the word to confirming.
- confirming and another score of at least 90 moves it to mastered.
- a miss stays where it is and comes back tomorrow. The word is not downgraded.
- a miss after mastered steps back to confirming and also comes back tomorrow.
The next review is never later the same day. The earliest nextReviewAt is tomorrow, so two passes cannot be recorded from two takes in one sitting. After mastered, the gaps are 21 days, then 90, then 365. A further pass stays at 365.
Due reviews are selected before any new word. New words come from the top 8,000 in BNC/COCA, more frequent bands first. If nothing is due and no new word is left, the client is told how many days until the next review. It does not invent a practice item.
The recording is bounded before it reaches Azure
Assessment audio is 16 kHz, 16-bit, mono. The cap is 15 seconds, which is also the upload cap: 44 bytes of WAV header plus one second of samples times 15. Recognition failures are mapped to a short message. NoMatch, initial silence, and babble each say what to change. They do not become a score of 0 mixed in with a real attempt.
The model recording is separate. It is Azure neural TTS with the target IPA written into SSML as a phoneme element, American English only. The voice the learner imitates and the score of their own take do not share a request.
What I would not do again
I would not store the raw assessment document on the client and score it there. The pass line, the mark line, and the next review date are the product. They stay on the server.
I would also not show AccuracyScore next to every phoneme. The learner needs the sound that failed, then another try. The number is for the scheduler.
Vocalace is the app this runs in: one American English word, the missed phoneme marked, and the review rules above. A new account can practice for 7 days.
Top comments (0)