DEV Community: Waldo

The ticket is the thinking

Waldo — Sat, 14 Nov 2026 13:17:42 +0000

Writing the ticket is the work, and everything after it is execution. That sounds backwards, because in most workflows the ticket is a light wrapper around the real work. In mine, the ticket is where I make the design decisions, name the edge cases and write down how the result will be verified. An agent builds what the spec describes, so the quality of the spec caps the quality of what comes back: a vague brief gets vague software. Over time the tickets turn into tests, the tests become context for the agent, and that context shapes the next ticket.

Every task is a ticket before it's a task

Before I open an editor, I write the spec as a document: what the user sees, what happens in the database, what the edge cases are and what "done" looks like. Only then do I open Claude Code. The spec forces the thinking to happen before implementation starts, and it draws a hard line around scope.

Scope drift is the real enemy. Most of the bugs I have introduced started with "while I'm in here, I should also fix X". When something adjacent comes up mid-session, I open a new ticket for it, and the current one stays clean.

Every bug becomes its own ticket

When you find a bug, the instinct is to fix it on the spot, while you are in the codebase and can see the cause. A bug found in passing is usually not fully understood, though. You see the symptom and you have a theory. Fix it inline and you fix it under time pressure, without writing down the failure mode or thinking through how to verify the fix.

I tested a bulk order on NfcPress, an NFC business card platform I built with a friend. Every card in the bulk PDF showed the same base design instead of its own. I could see what was probably wrong, and my first impulse was to fix it there and then. Instead I opened NFC-57.

Writing the ticket forced a proper diagnosis, and it turned up two separate bugs. The card design was generated in the background after the page had already responded, and Vercel stopped that job before it finished. When cards were rebuilt, they were sorted by the time they were created instead of by their code, so the order could change from one run to the next. Both root causes were documented before a line of code changed.

The ticket also showed that no automated test covered this. As soon as NFC-57 closed, I opened NFC-58: a Playwright test with three scenarios designed to catch these failure modes if they regress. The bug became a ticket, the ticket a fix, and the fix a test that now runs before every deploy. It has not had to catch a regression yet. The proof comes the next time something breaks and this test fails first.

The kanban is a view, not the system

I have a kanban board with Backlog, In Progress and Done, and it gives me a quick read on what is moving. The system lives in the .md files behind it. Every ticket is a markdown file: frontmatter for status, then a spec with context, scope, what to build and how to verify it. A sync daemon generates the board from those files, so if the board disappeared I would lose the view and keep the knowledge. The files stay, and I can grep them, count them and reference them from other tickets.

📄 The future of AI agents looks a lot like 2010.

Tickets accumulate

After a wave of features ships, I run a retro: a pass through the closed tickets to work out what they imply, taken together, about the product's correctness. After roughly 38 NfcPress tickets had closed, I opened a ticket whose only job was to read the other 38. Its spec began like this:

As of 2026-05-22, many product behaviors have been shipped. None of these are covered by regression tests. Any future change could silently break them.

That ticket read the closed history and produced two outputs. The first was a set of new tests for evals/smoke.ts, the HTTP checks that run before every deploy. The second was a set of additions to evals/product-evals.md, a manual golden checklist for behaviour that needs a browser and human eyes to verify. A golden is a one-line statement of required behaviour that must stay true across releases.

The next retro came after NFC-39 to NFC-48, and its spec picked up where the first one stopped:

The previous evals round covered up to roughly NFC-38. Since then the following tickets shipped and are not yet covered by any test or golden...

It then listed every feature, each with the specific smoke tests and goldens to write for it. Tickets ship, a retro reads the history, and the goldens and smoke tests it writes go into the harness that protects every later ticket. Each wave makes the harness stronger, and nothing evaporates.

Diagram: Every task starts as a written ticket, and an agent builds to that spec. A bug found on the way becomes a new ticket. After a wave of closed tickets, a retro reads the history and writes goldens and smoke tests, and every next ticket starts with them in context. (see the diagram in the original post)

Evals feed back into execution

Every new ticket opens with the smoke suite and the goldens already in context. Claude implements the feature against the new spec and against everything the last 48 tickets learned. NFC-58 made this explicit: its context section carried a table inherited from NFC-57.

Failure	Symptom	Root cause
`design_front_path = null` on all cards	Every PDF page shows identical base design	`void generatePerCardDesigns()` got killed before completing
Sharp crash (float dimensions)	Same as above, silent	`qrConfig.width` is a float; `Math.round()` missing
Wrong page order	Card 2 appears on PDF page 1	Cards were sorted by creation time; now sorted by `code`

Known failure modes from the bulk PDF bug are now executable specs, sitting in the context the agent reads first. Bugs teach the goldens, the goldens constrain the next implementation, and that implementation produces new bugs and new goldens.

None of this requires a methodology. It requires one discipline: write the ticket before doing the thing. After 48 NfcPress tickets, that discipline means the next ticket starts with every failure the earlier ones wrote down.

Originally published on waldo.vanderlore.de.

The future of AI agents looks a lot like 2010.

Waldo — Sat, 14 Nov 2026 13:11:42 +0000

The most useful thing I have added to my agent setup is a kanban board. It did more for the results than any new model or better prompt. The board is three Markdown files and an Obsidian plugin. It decides what Claude Code works on, when a piece of work counts as done, and when it comes back to me for a decision. Each of those rules comes from the backlog, the user story and the acceptance criteria that software teams have used for years, now applied to a team of agents.

Letting the agent run

The conversation around AI agents is almost entirely about orchestration: tool use, memory layers, multi-agent routing. Almost none of it is about managing the work, and my own setup had the same gap. I would open a Claude Code session, pick something to build and let the agent run.

Sometimes that worked, and often it drifted. Context would bleed from one session into the next. When I came back three days later, I had no idea what state things were in or why something had been built the way it was. The problem was never the agent: I had stripped every management structure out of the workflow and expected nothing to change.

The kanban board

The board file

The board is one Markdown file that the Obsidian Kanban plugin renders as three columns: Backlog, In Progress and Done. Each card is a wikilink to a story file, and nothing else goes on a card. Two cards waiting and one finished look like this:

## Backlog
- [ ] [[CVR-019-bedrock-card-rendering|CVR-019 Replace card rendering]]
- [ ] [[CVR-020-dashboard-rollout|CVR-020 Dashboard full rollout]]

## In Progress

## Done
- [x] [[PRJ-008-deploy-pipeline|PRJ-008 Deploy pipeline refactor]]

The story files

Every card has one story file, named CVR-NNN-slug.md. Here is one, abridged:

## CVR-019: Replace card rendering
Goal: cards render from the new classifier output; old path deleted.
Hard gates:
- [ ] all golden cases render identically
- [ ] old renderer removed, not flagged off
Blocker: none
Work log:
- session 1: classifier clean in shadow mode, opening render work

Hard gates are the definition of done, and the story does not move until every one of them is checked. When something cannot proceed, the card stays in Backlog marked ⚠️ BLOCKED. The agent does not skip it silently or abandon it: it marks the card, explains the blocker and stops.

Who moves the cards

I ask Claude to tackle the next story. Stories move to In Progress as the agent works, and they come back to me when they need a human call. Claude is the only thing that moves cards, which is why the board cannot lie about the state of the work. Nothing gets worked on without a story.

Before a story moves to Done, there is a context update step. The work log and the project's CLAUDE.md are the place to capture what was decided and why, on top of what was built. A story with zero checked boxes is not valid, and durable learnings don't survive on vibes.

The team

The team is a Markdown file of named roles, each called at a specific point in the process. The most useful one is /refinement-team-deep, the planning council. Six sequential agents go through a plan: Critic, Contrarian, Tech Lead, Software Architect, Engineering Manager, and finally a Revised Planner who synthesises the other five. Each one addresses a flaw the previous one didn't.

Two stories instead of one

I was redesigning a campaign analysis pipeline when the council flagged a problem with my plan. Shipping the new classification layer and the rendering change together would make it impossible to isolate regressions. The revised plan split them into separate stories: ship the classifier first in shadow mode, verify it on known cases, and only then open the rendering work. It cost two minutes of latency, and the output was two stories instead of one, each with clearer hard gates.

The other roles

The Orchestrator is the default mode: it reads the board, routes work and breaks epics into stories. The Tech Lead is called before anything architectural ships, and the Designer runs /ux-review before a line of markup is written. For a fast single pass, the Critic runs /critique and returns PROCEED, REVISE or ABORT in under a minute. Together with the board, that gives every story the same path from plan to done.

Diagram: How a story moves through my agent team: the council refines the plan into a story with hard gates. The agent builds it, the roles review it, and the hard gates are checked. An open gate sends it back to building, a blocked story comes back to me, and only after a context update is it done. (see the diagram in the original post)

The retro

The next SDLC primitive I am working on is the retro, with two experiments in progress. A weekly review skill gives me a structured moment to step back, reflect on how the week went and carry the useful parts forward. An agent retro skill looks for patterns across sessions: which skills I reach for, which tasks get repeated or abandoned, and where the gaps are. Both are still rough. The retro is the one SDLC ritual whose job is turning mistakes into process, and agents cannot yet do that for themselves.

Where this is going

Backlog, stories, acceptance criteria, roles and reviews: we have been working this way for fifteen years. AI agents did not invent a new way of working; they gave us a new team to manage. That is the bet I am making. The SDLC patterns transfer, and the more closely we follow them, the better agent-driven development gets.

The open problems remain. Context does not fully survive session boundaries, and decision rationale evaporates from transcripts. Shared context across sessions and collaborators is unsolved. At AI Bar Camp Berlin last week, every room circled back to the same questions, and nobody has cracked it.

My guess is that the answers look a lot like the questions: better story handoffs, better session summaries, and better ways to make the work log the source of truth. That would mean deeper investment in the old practices rather than inventing new ones, though I am not sure yet. Until then, the part of my setup that keeps a team of agents on track is a board of three Markdown files, embarrassingly low-tech and doing the job it always did for human teams.

Originally published on waldo.vanderlore.de.

Write the eval before the fix: building a WhatsApp transcriber with LLM-TDD

Waldo — Sat, 14 Nov 2026 13:10:54 +0000

Voice messages are my pet peeve. They arrive while I am in a meeting or putting my daughter to bed, and I want the chance to read them rather than stop everything and listen. They pile up instead: a friend's two-minute update waits until the evening, and by then half of it is old news. Not wanting to do to others what I do not want done to myself, I would thumb-type long messages, wishing I could just send a quick voice message instead.

WhatYouSay fixes both sides. It is a small server linked to a WhatsApp account. When you send a voice note, it gets transcribed, cleaned up and posted right under the note in the same chat, so the other person can read instead of listen. I have started sending voice notes again, and the friends who use it send me notes I can read between meetings.

The idea comes from WhatScribe, a hosted service that does the same thing. I wanted my own version with guaranteed privacy, and I wanted to decide what it keeps.

I must admit, it took a lot more than plugging in Whisper. My friends and I talk in English, German and Dutch, some of us mixing all of them. And voice notes are private, so the server had to forget them once the transcript was sent.

How I build: LLM-TDD

LLM-TDD is test-driven development where an LLM writes the code and I write the checks first. I define what good looks like, turn it into evals, let the model build against them, and only then use the result live. Model output is not deterministic, so the eval compares against a reference instead of an exact string.

Diagram: LLM-TDD: I define good, I write checks, the LLM writes code, I run real examples and compare with a reference. A mismatch sends me back to the code; a release decision puts it live; a live failure becomes a new check. (see the diagram in the original post)

The loop starts with me defining what good looks like and writing the checks that test for it. Then the LLM writes the code, I run real examples through it and compare the output with a reference. A mismatch sends me back to revise, either the code or a prompt, until the comparison passes. Only a release decision puts the change live, and a failure in the live system becomes a new check.

In WhatYouSay the code under test is a pipeline. Whisper transcribes each voice note, in two passes for longer ones. A second model cleans up the text, translates it if asked and summarises long notes. The real examples are voice notes in English, German and Dutch, from me and the friends who use WhatYouSay, and for 20 of them the reference is a ChatGPT transcript of the same audio. The next sections show what that loop caught.

Twenty notes against a reference

I scored the pipeline against ChatGPT transcripts of the same 20 voice notes, and the first scored run matched 69.9% of the words. Most of the gap came from the cleanup model rewriting how we actually talk. German "gucken" became the more formal "schauen", and "matcht" became "entspricht". It deleted "ja", "also", "mal" and "halt" as filler, though in German those words carry the tone. It also translated English tech terms that a German speaker had chosen to say in English.

Each of those edits looks like an improvement on its own, and only the comparison with a reference marked them as errors. The first fix was a prompt that lists what never to touch: colloquialisms, regional words, discourse particles and embedded English. On the same 20 notes, the match went to 86.4%. It was a first step, not the end state: the gates, repairs and fallbacks in the next sections came after it, and every one of them has to pass the same notes.

Word match against reference transcripts, same 20 voice notes


Before prompt revision	69.9%
After prompt revision	86.4%

The first fix: the cleanup prompt lists what it must never touch, such as colloquialisms and mixed-language speech.

One gate per language

After that fix, each language needed its own gate. German drafts may change 8% of the words; past that, the pipeline stops trusting the draft, keeps Whisper's words and borrows only the formatting. English gets 5%, plus an order to keep every "like", "you know" and "I mean", and never to add helper words that make a sentence smoother than it was spoken.

Dutch gets no word-count gate, because the Dutch prompt writes spoken numbers as words and that alone moves the count. It got two narrow repairs instead. In a note about music and dancing, Whisper heard "prapjes" where the speaker meant "grapjes" (jokes), one consonant away. The model kept choosing "praatjes" (chats), which fits the topic and sounds further off. The Dutch rule now allows exactly one misheard consonant, and only when the sentence makes the word clear. In another note, about a birthday boat trip on the Spree, the cleanup turned the river into "spreeuwen" (starlings), so the Dutch prompt now names the Spree.

The test that stays red

Some repairs still miss. In one German test note Whisper hears "gelaschten" where the speaker said "gelöschten" (deleted), and the cleanup "fixes" it to "gesendeten" (sent): a real word with the wrong meaning. The loop keeps known failures recorded as red tests, so that test stays red on purpose until a change turns it green.

Whisper stops early

Whisper has a second failure mode: it quits halfway through a note. A 32-second English note came back as 27 words, and at first I blamed randomness. The real cause was the server passing the sender's display name to Whisper as context. Whisper treats its prompt as audio it has already heard, so it found a phonetic match for the name mid-note and stopped there. Without the name, the same audio came back as 86 words.

Whisper also copies the style of its prompt: a lowercase list of names made it return lowercase, unpunctuated text and drop 21 of 226 words. The names I configure for correct spelling now ride only on pass two, and pass two is thrown away unless it keeps 80% of pass one's words. Healthy passes keep 91 to 100%, while every truncation I measured kept 33 to 55%, a gap wide enough for a fixed line. If both passes come back thin, below 1.2 words per second, a third pass on a different Whisper model takes over. When names and completeness conflict, the pipeline keeps the full transcript and drops the names.

How the pipeline works

Since 11 September, WhatYouSay has handled 230 voice notes in three languages, three hours of audio in total, and the longest ran five minutes. Each note is downloaded into process memory and runs through four steps, which now carry the rules from the sections above.

1. Transcription: Groq Whisper large-v3

Pass one runs with no prompt and detects the language. Notes under 15 seconds usually stop there, and that is a quarter of them (61 of 230). Longer notes get a second pass with the detected language forced, so a German note with one English phrase does not flip language halfway through. Whisper also hallucinates leftovers from its training data, such as "Subtitles by the Amara.org community" after a silent stretch, and those get stripped first.

2. Cleanup: a Groq-hosted LLM

Raw Whisper output is readable but not pleasant: filler words, no paragraph breaks, merged words like "Freitagzeit" for "Freitag Zeit". A second model fixes punctuation, adds paragraph breaks at topic shifts and repairs clear mishearings, within the never-touch list and the language gates. Notes under 30 words skip this step.

The output has to pass a check. If a note over 50 words comes back less than half as long, or a note over 80 words comes back as one wall of text, the pipeline retries once with a precise correction. If that fails too, it sends Whisper's raw text.

3. Translation, if turned on

If the detected language differs from the target, another call translates the cleaned text. If translation fails, the original language wins.

4. Summary for long notes

Above 120 words, the reply can open with a summary of at most 60 words in the same language. The full transcript stays underneath. In the screenshot, a 49-second ramble about oat milk is cut to three sentences on top of the clean transcript.

What the checks protect

Every fallback in the pipeline points the same way. A cleanup that cuts too much is replaced by Whisper's raw text, because losing part of a message is worse than leaving it messy. A failed translation hands back the original language, because a bad translation should never replace a readable transcript. Configured names that cost words are dropped, and a German draft that drifts past its gate loses its wording to Whisper's. The evals hold that order in place: a prompt or code change has to pass the same reference notes before a release decision puts it live.

No transcript history

Decide what your software must retain before you decide what it can collect. Once the reply has landed in WhatsApp, the pipeline needs almost nothing. I have no use for a searchable archive of my voice notes, and I do not want a database that slowly turns into a record of private conversations. Storing transcripts would have made search and history easy, and left me with an archive of 230 private voice notes that were only ever meant to be read once. The app therefore never writes transcript or summary text to its database.

Each note gets one metadata row: sender, duration, language, token counts, a WhatsApp message ID and a timestamp. The transcript and summary columns are written as NULL and the row is flagged content_deleted = 1. The schema still has those transcript-shaped columns, so it is the write path that keeps them empty. After 90 days the sender goes too: the name is wiped and the WhatsApp ID becomes a keyed hash. The usage numbers in this post come from these rows alone, and not one of the 230 contains transcript text.

What leaves the server

The audio and the derived text go to Groq for transcription and text processing. The reply goes back into WhatsApp. While the dashboard is open, the transcript also streams to that browser over a live SSE connection. A short-lived in-memory cache holds each sent reply so WhatsApp can re-send it when a recipient's device asks.

The content clearly leaves the machine, and every transcribed note passes through a third-party AI API. What WhatYouSay guarantees is narrower: no durable transcript archive once the job is done. On my account it only transcribes the notes I send, so the audio that goes to Groq is my own voice.

The boring architecture

WhatYouSay runs as one long-running Node.js and Express process on a small Oracle Cloud ARM server in Frankfurt. The database is SQLite through Node 22's built-in node:sqlite, login works with magic links sent through Resend, and Cloudflare sits in front. Baileys, an unofficial WhatsApp Web linked-device library, holds the WhatsApp connection. It stays one process because the WhatsApp socket, the live dashboard connections and the reply cache all live in memory. Serverless infrastructure and horizontal scaling for a personal tool would be a second project.

Three things that bit me

When a recipient's device cannot decrypt a WhatsApp message, it asks for a re-encrypted copy. Ignore that retry receipt and the person sees "Waiting for this message." forever. Baileys gives you the hook, but you still need a short-lived cache of sent replies, and I learned that in production rather than in a neat local test.

SQLite's write-ahead log was the second. A running server and a migration script writing at the same time can leave it in a bad state, while both writes look successful. Two of three imported users disappeared on the next restart. Now I stop the server before any external script touches the database.

The third was a retired model. On 21 September Groq decommissioned the model my cleanup ran on, and nothing in my code changed. Every note longer than a sentence stopped getting a reply, while short ones kept working because notes under 30 words never call the LLM. It looked like the server was down, although Groq had announced the retirement four weeks earlier. The server now sends me a Telegram alert when a model disappears or a step degrades. That is the same loop running in production: the live failure became a check.

Run your own

The code is on GitHub: whatyousay-oss, MIT licensed, with a step-by-step guide for a free Oracle Cloud server. It is invite-only by default. The public copy is a fresh repository with its own history. WhatYouSay runs on an unofficial WhatsApp library, which sits in a grey area of WhatsApp's terms.

The evals I write at my desk cover the failures I can picture in advance. The retired model and the retry receipts showed up in production, where the first symptom was a reply that never arrived. In LLM-TDD that is where the next check comes from: a failure in the live system goes back into the evals. The next time, a test or an alert catches it instead of the person waiting for the text.

Originally published on waldo.vanderlore.de.