DEV Community: Nikolai | ExamReady

Generating realistic exam listening tests with TTS: voices, pauses and timestamps

Nikolai | ExamReady — Sat, 17 Oct 2026 20:59:51 +0000

Listening is the part of a language exam where you can't slow down. The recording plays, you answer, and the next one starts. If you're building practice material for it, a plain audio file isn't enough: it has to behave like the real thing, including the instructions, the pauses to read the questions, and the second play.

I'm building ExamReady, a platform for preparing for German B1/B2 exams, and listening practice is one of the parts I had to figure out from scratch. This post is about how I generate the audio with text-to-speech, assemble it with ffmpeg, and get timestamps out of the build so the app can show transcripts and highlight text while the audio plays.

ExamReady is an independent project and is not endorsed by ÖSD, ÖIF or the Goethe-Institut.

The problem

I needed listening practice that:

has several speakers who sound different from each other
includes the exam framing: instructions, time to read the questions, and a second play
lets the learner see a transcript afterwards, and ideally see which line is being spoken right now

I didn't want to hand-edit audio in a DAW for every recording. Every change to a script would mean re-exporting by hand, and I'd lose track of what was where. I wanted the audio to be a build artifact, produced from a text script.

Step 1: describe the recording as segments

Each recording is a script made of segments. A segment is either speech (a voice and some text), a silence, or a chapter marker.

type Segment =
  | { kind: "speech"; voice: string; text: string }
  | { kind: "silence"; seconds: number }
  | { kind: "chapter"; title: string };

const recording: Segment[] = [
  { kind: "chapter", title: "Instructions" },
  { kind: "speech", voice: "narrator", text: "You will hear a short conversation twice." },
  { kind: "silence", seconds: 5 }, // example value
  { kind: "chapter", title: "Conversation" },
  { kind: "speech", voice: "speakerA", text: "..." },
  { kind: "speech", voice: "speakerB", text: "..." },
];

The important idea is that everything the learner hears is in this list, including the silences. Silence is not something I leave to chance or to the TTS engine. If the exam gives you time to read the questions, that time is an explicit segment in the script.

Voices are just names mapped to a text-to-speech voice configuration, so a conversation between two people uses two different voices, and the narrator uses a third.

Step 2: generate speech per segment

For speech I use ElevenLabs text-to-speech. Each speech segment becomes its own audio clip. Here is the shape of the build step, in pseudo-code:

clips = []
for seg in script.segments:
    if seg.kind == "speech":
        clips.append(tts(voice=VOICES[seg.voice], text=seg.text))
    elif seg.kind == "silence":
        clips.append(make_silence(seg.seconds))
    # chapters produce no audio, only a marker

Generating clip by clip, instead of one long request for the whole conversation, has a practical benefit: I control exactly where each voice starts and stops, and I can insert silence between any two lines.

Step 3: assemble with ffmpeg and record the timestamps

Once I have the clips, ffmpeg concatenates them into a single file. The part that turned out to matter most is that the build also prints the start time of every segment.

To do that, the build measures the duration of each clip and keeps a running total:

t = 0.0
timeline = []
for seg, clip in zip(script.segments, clips):
    timeline.append({"segment": seg.id, "start": round(t, 2)})
    t += duration_of(clip)  # measured from the real audio, not estimated

The key detail is "measured from the real audio, not estimated". If you guess durations from the text length, the timestamps drift and the highlighting looks wrong after the first few lines. Reading the actual duration of each generated clip keeps the timeline honest.

Then the concatenation itself is a single ffmpeg call over a list of the clips in order:

ffmpeg -f concat -safe 0 -i clips.txt -c copy listening.mp3

Using stream copy only works if all clips share the same format and parameters, so it's worth making the silence clips match the speech clips. If they don't, re-encoding is the safer fallback.

What the timestamps are used for

The printed start times become data attached to the recording. The app uses them in two ways:

Transcripts. After the exam-style run, a learner can read the transcript, line by line, with each line linked to its position in the audio.
Highlighting. While the audio plays, the app compares the current playback time to the start times and highlights the matching text.

Because the timeline comes from the same script that produced the audio, the transcript and the audio can't drift apart. If I edit a line and rebuild, everything is regenerated together.

Baking in the exam framing

I put the exam framing into the file itself: the instructions, the reading pauses and the second play. That was a deliberate choice. A learner doing practice should experience the same rhythm as in the exam, not a clean recording with the pacing left to them.

The tradeoff is that the file is less flexible. If I wanted a version with no pauses, I'd have to build a second file. For exam practice I think that's the right tradeoff, and since it's all generated from the script, a second build isn't much work.

What I'd watch out for

A few cautions rather than hard-won certainties:

TTS voices are not real exam audio. They're clear and consistent, which is good for practice, but real recordings can have different accents, speeds and background noise. I wouldn't claim generated audio is identical to what a learner will hear on exam day.
Check pronunciation. Names, numbers, dates and times matter a lot in listening tasks, and TTS can read them in a way you didn't expect. It's worth listening to the output instead of trusting it.
Keep the script as the source of truth. Don't patch the audio by hand. If something is wrong, fix the script and rebuild, otherwise the timestamps stop matching.

Takeaways

Describe the recording as data: segments with a voice and text, silences and chapters.
Make silences explicit. They are part of the exam.
Measure clip durations from the generated audio, never from the text.
Print the timeline from the build. Transcripts and highlighting come almost for free.
Listen to the output. A pipeline that runs without errors can still read a number wrong.

If you've built something similar, for exams or for any audio that needs to stay in sync with text, I'd be curious how you handled the timeline.

I'm building this at examready.at, feedback welcome.

Don't let the LLM do the maths: grading writing with AI but scoring in code

Nikolai | ExamReady — Sun, 11 Oct 2026 22:03:24 +0000

If you ask a language model to grade a text and also to count the words, add up the points and apply the scoring rules, it will usually give you a confident answer. Sometimes the answer will be wrong, and you won't notice, because the output looks reasonable.

I ran into this while building the writing and speaking practice in ExamReady, a platform for preparing for German B1/B2 exams. The setup I ended up with is simple to state: the LLM judges, and my code does everything that can be computed. This post explains where that line is and why I drew it there.

ExamReady is an independent project and is not endorsed by ÖSD, ÖIF or the Goethe-Institut.

The problem

In a writing exam, a learner gets a task, writes a text, and is graded on several official criteria. There are also hard rules around the result. One example: if the text has less than 50% of the required number of words, it scores 0 points.

That rule has nothing to do with language quality. It's arithmetic. A text can be well written and still score 0 because it's too short.

The tempting approach is to put the whole grading guide into one prompt and let the model produce a final score. That's one API call and one result, which is attractive when you're a solo developer. But it mixes two very different jobs.

What models are good at, and what they aren't

A model is good at judging things that need language understanding. Does this text address the task? Is the register appropriate? Is the grammar mostly correct, and where does it go wrong? Those are the questions I want an LLM for.

A model is unreliable at counting and at arithmetic. Counting the words of a text, deciding whether that is above or below half the required number, and summing criterion scores into a total are all things that a few lines of code do exactly, every time. Asking a model to do them gives you a result that is usually right, and occasionally wrong in a way that is hard to detect and impossible to reproduce.

So I split the work.

The split

The LLM (Gemini) gets the task and the learner's text, and returns grades per official criterion, plus written feedback. It does not return a total and it does not decide whether the text was long enough.

The code then:

counts the words in the learner's text
applies the official rules, for example 0 points if the text is under 50% of the required words
combines the criterion grades into the final result

Here is the shape of the model output I ask for:

type CriterionGrade = {
  criterion: string;     // one of the official criteria
  points: number;        // within that criterion's allowed range
  comment: string;
};

type ModelGrading = {
  grades: CriterionGrade[];
  feedback: string;
  corrections: { original: string; suggestion: string }[];
};

And the part that belongs to the code:

function countWords(text: string): number {
  return text.trim().split(/\s+/).filter(Boolean).length;
}

function finalScore(text: string, task: WritingTask, g: ModelGrading) {
  const words = countWords(text);

  // rule applied in code, not by the model
  if (words < task.requiredWords * 0.5) {
    return { total: 0, reason: "too_short", words };
  }

  const total = g.grades.reduce((sum, c) => sum + clamp(c.points, task), 0);
  return { total, words };
}

Two small details matter here. First, the word count is a function I can unit-test. I can feed it texts and check the result, which I can't do with a prompt. Second, the model's points are clamped to the allowed range for each criterion, because a model can return a value outside what the scale permits. I treat model output as input that needs validation, not as a verdict.

Verifying corrections against the learner's text

The second place where I stopped trusting the model blindly is corrections. A useful feedback feature is showing the learner a specific mistake in their own sentence and a better version. The failure mode is that the model quotes something the learner never wrote, or slightly rewrites their sentence before "correcting" it.

So every correction is verified against the learner's own text before it is shown. In essence:

function verifiedCorrections(text: string, list: ModelGrading["corrections"]) {
  return list.filter((c) => text.includes(c.original));
}

If the quoted passage isn't actually in the text, the correction is dropped. A missing correction is better than a wrong one pointing at a sentence the learner didn't write. The real check can be a bit more tolerant about whitespace, but the principle is the same: the code checks the model's claims against the source.

Why not do everything in the prompt?

I can see the argument for it. Modern models are good, and often they'll count correctly. But "often" isn't the standard I want for something that decides whether a learner thinks they would pass. A few reasons I prefer the split:

Reproducibility. The same text always gets the same word count and the same rule application. Only the judgement part varies.
Testability. Rules in code can have tests. Rules in prompts have vibes.
Debuggability. When a score looks wrong, I can tell whether the rule or the model's judgement caused it.
Rule changes. If a rule changes, I change a function, not a prompt I then have to re-validate.

The cost is more code and a more structured output from the model. I think that's cheap for what it buys.

A note on speaking

Speaking practice uses a live AI conversation partner built on Gemini Live. It's a different shape of problem, since it's a conversation rather than a document, and it costs me roughly ten cents per speaking task. I'm still learning what the right boundary is there, so I'll keep this post about writing.

Takeaways

Let the LLM judge language, and let code count, compare and add up.
Treat model output as untrusted input: validate ranges and shapes.
Apply official rules in code so they can be tested and reproduced.
Verify any quote or correction against the learner's own text before showing it.
Prefer a missing piece of feedback over a wrong one.

None of this is clever. It's mostly about not asking a language model to do what a function does better.

I'm building this at examready.at, feedback welcome.

I built an exam prep platform for German B1/B2 exams as a solo founder: what I learned

Nikolai | ExamReady — Fri, 09 Oct 2026 07:44:42 +0000

I'm a solo developer, and for a while now I've been building ExamReady, a platform for preparing for official German exams. Most of the people it's meant for are immigrants who need to pass a B1 or B2 exam.

This is a build-in-public summary: what the product is, how it's put together, and the things I got wrong along the way. It's not a success story. It's a list of decisions, and a few of them I'd now make differently.

ExamReady is an independent project. It is not official and is not endorsed by ÖSD, ÖIF or the Goethe-Institut.

The problem I wanted to solve

The exams I cover are the ÖIF Integration Exam B1, ÖSD B1 (ZDÖ), ÖSD B2 and the Goethe-Zertifikat B1. Each has its own format, task types and scoring rules. Preparing means learning German and learning the exam format at the same time.

My idea was simple: keep the exam material in German, because that's what the real exam looks like, but explain everything around it in the learner's own language. The explanation of a task type should not itself be a reading test.

What exists today

There are 4 courses with 104 short interactive lessons, 20 full model tests, and explanations in 7 languages: German, English, Russian, Ukrainian, Turkish, Serbian and Romanian. It's free to start (2 keys a day), and Pro is a one-time payment rather than a subscription.

The stack

I picked it to be boring on purpose:

Next.js frontend
NestJS backend
MongoDB
Docker Compose behind Caddy on one small VPS

I'm the only person who deploys and debugs it, so I optimised for being able to hold the whole system in my head. One server, one compose file, one reverse proxy. I'm not claiming this setup is the right one for everyone, only that it fits one person maintaining a product.

A content model that doesn't fight me

The first real design decision was how to store exam content. I ended up with a normalized model: Exam > model test > module > task. Every task has a type, and the answer key lives separately from the task.

type Exam = { id: string; tests: ModelTest[] };
type ModelTest = { id: string; modules: Module[] };
type Module = { id: string; tasks: Task[] };
type Task = { id: string; type: TaskType; content: unknown };

// stored separately from the task
type AnswerKey = { taskId: string; value: unknown };

Keeping the key apart has two benefits. The client never has to receive answers it doesn't need to display, and I can change or regenerate a key without touching the content.

Scoring goes through a registry: one scorer per task type, looked up by the task's type.

const scorers: Record<TaskType, (answer: unknown, key: unknown) => Score> = {
  // multipleChoice: ..., trueFalse: ..., and so on
};

function score(task: Task, answer: unknown, key: AnswerKey): Score {
  return scorers[task.type](answer, key.value);
}

Adding a new task type means adding a type and a scorer. That's the whole contract. It kept me from scattering ad hoc scoring logic across the codebase.

Lessons as decks

Lessons are full-screen decks: one idea per slide, in the style of Brilliant. About 60% of a lesson is teaching, there are 3-4 questions, and a realistic exam task comes at the end. The one-idea-per-slide constraint forces me to cut. If an explanation doesn't fit on one slide, it's usually two ideas, or I haven't understood it well enough yet.

Where I got things wrong

A few honest ones.

The answer keys were guessable. I noticed that answer patterns repeated between tests, and that true/false keys alternated too regularly. A learner could guess the pattern without understanding anything. Now key sequences are generated with constraints (different from other tests, no strict alternation) and a script checks them.

Manual testing wasn't enough. Clicking through lessons yourself mostly covers the path you already know. I now run AI agents acting as personas through every lesson, plus an automated "impatient user" sweep. They found real bugs: a second example that never showed up, a writing draft lost when going back, and links losing the language prefix. I'll write about that setup in a separate post.

Translations needed real review. Machine-like translations had to be reviewed and rewritten. And when I removed em dashes from the text, some Russian and Ukrainian sentences broke, because those languages use the dash in place of the verb "to be".

Where I use AI, and where I don't

Writing and speaking practice use AI. For writing, the LLM (Gemini) gives grades per official criterion and written feedback. My code counts the words and applies the official rules, for example that writing under 50% of the required words scores 0 points. Models are unreliable at counting and arithmetic, so I don't ask them to do it. Corrections are also verified against the learner's own text before they're shown.

For speaking, there is a live AI conversation partner built on Gemini Live. It costs me roughly ten cents per speaking task, which is a real number I have to keep in mind when I think about pricing.

Building with AI coding agents

I built all of this as a solo developer with heavy use of AI coding agents (Claude Code). It let me cover a lot of surface area: content tooling, frontend, backend and scripts. It didn't remove the need to decide what to build, to read the diffs, or to test like a real user. The amount of code I can produce went up. The cost of judgement didn't go down.

Takeaways

Keep the stack boring when you're the only maintainer.
Model your content properly early. Separating answer keys from tasks paid off.
Don't let the LLM do arithmetic or apply rules. Let it judge, let code count.
Test as an impatient, confused user, not as the author.
Treat translations as content that needs review, not as a switch you flip.

If you've built exam prep or other structured-content products, I'd like to hear what you'd do differently.

I'm building this at examready.at, feedback welcome.