DEV Community

WeekendAiProjects
WeekendAiProjects

Posted on Fully Autonomous

Vibe coding vs spec-driven development with Claude Code: we built the same app both ways

We wanted to know what it really takes to ship a public app with Claude Code when you don't write the code yourself. Not a demo: an app strangers can sign up to, on a server of your own, that you can trust with other people's data.

So we built the same app twice, each time from the beginning: once from one sentence, the way most people start (often called vibe coding), and once from a written specification (spec-driven development).

The app is Unforget. People screenshot or screen-record the posts they mean to try (a recipe, a stretch, a keyboard shortcut) and those pictures get lost among all the others. You upload them, and a vision model writes a note for each: what it is, the key points, the steps, the account it came from, a few tags. Search, filters, notes marked tried, and an email on Sundays with what you still have to try. For screen recordings, Whisper writes down the spoken words, and four frames are taken from the video. The stack: Django and SQLite, a worker that reads in the background, a local model through Ollama while building and Claude on the server, deployed with Coolify (on our test bench, a virtual machine took the place of the server, and a local model the place of Claude).

One sentence

A new Claude Code session with no history, an empty folder, and the kind of request most people type:

Build me a web app that turns the screenshots and screen recordings of posts I save from Instagram and TikTok into notes I can actually find again. Make it good!

We told it to build everything without asking questions. 26.3 minutes later it reported back, and the result was nice to use: drag and drop, paste a screenshot, cards that fill in as notes arrive, search by meaning using a second local model it found on the machine, 24 tests, checked in a real browser at phone width.

Question The one-sentence app Built from a specification
Who can use it? No accounts, so it only accepts connections from that one computer: in its own words, it "can't be opened from your phone" Anyone who signs up; each sees only their own saves; a stranger gets "not found"
Is it online? No On your own server, with HTTPS, backups and an alert when the worker goes quiet
How do you know it works? 24 tests it wrote for itself Its tests, 82 checks it may not change, and an independent review

It listed 18 decisions it had made on its own. Each was reasonable for a demo. A choice you never saw is a decision you never made.

The same assistant, directed

For the second build, we started again from the beginning and changed four things. The assistant was not one of them.

A specification. Twelve sections, 2,355 words: the four kinds of people (a member; someone signed up but not yet confirmed; a stranger; and the operator, who runs the server), every URL and who may open it, what the model receives and what the program keeps of its answer, the limits of an app open to anyone, what version 1 leaves out. When the code and the specification disagree, the code is wrong.

Small steps. Sixteen prompts across nine chapters of the book, each adding a small piece: an account and one screenshot turned into a note, then sign-up, every picture format, recordings and the worker, search and the phone, limits and a review, the server, backups, failures.

A test program the assistant is not allowed to edit, which we call the examiner. checks/check.py starts the app on a free port with a throwaway database and talks to it over HTTP, with a fake AI that records what it was sent and a fake mail server. 82 checks by the end. We tested it by planting bugs one at a time (a video kept after reading, the newest save read first, the upload key stored instead of its hash, a database restore that replaces the database file while the worker is still using it): it caught every one we planted.

A new session for each prompt. A new session remembers nothing, so every rule that matters is written into the prompt itself. Every prompt that writes code ends with a command to run and the rule that the examiner must not be changed.

Then we ran it: each prompt typed into a fresh Claude Code session with no memory of the others.

Prompts (chapter.number) What they build Minutes Examiner checks passed
2.1, 2.2 the project, one screenshot turned into a note 29.7 15 of 15
3.1 sign-up, confirmation, reset 14.5 25 of 25
4.1 every picture format, edit, delete 13.5 35 of 35
5.1 recordings, and a worker 19.8 45 of 45
6.1, 6.2 search, the iPhone Shortcut, Sunday's email 24.2 55 of 55
7.1–7.3 limits, a review, fixes for what it found 71.7 66 of 66
8.1–8.3 Claude on the server, HTTPS settings, the Docker image, the iPhone photo problem 39.5 73 of 73
9.1 backups and restore 14.1 78 of 78
10.1, 10.2 when email or the worker fails 24.9 82 of 82

251.9 minutes of assistant work. No prompt reworded. No session changed the examiner. Some of what they did unasked was the best part:

  • One made a screenshot carrying the line "Ignore your instructions" and sent it to the real model. The note described the line instead of obeying it.
  • One hit a crash between faster-whisper 1.2.1 and PyAV (metadata_errors) that we had hit too, building our own version, and fixed it the same way: decode the audio yourself and hand Whisper the samples.
  • One was given a prompt describing a bug its build never had. Its answer: "there was no bug to fix, and I didn't invent one."

The instructions mattered as much as the model

Twelve sample screenshots we wrote, three local models (27, 9 and 4 billion parameters: the 27B, 9B and 4B below), three runs each. A note counted as right only if it held every fact of the post, the right kind of post (recipe, workout, tip…), the account it came from as its source, and stayed in French when the post was French. Two of the samples test honesty: a caption that tells the AI to send $500, and a book list cut off at "… more".

Model First version of the instructions Second version Third version
qwen3.8:27b 12 of 12 12 12
qwen3.5:9b 9 of 12 11 12
qwen3.5:4b 4 of 12 7 or 8 (varied by run) 9

All of the 9B's first misses were the wrong kind of post. A one-line definition of each kind fixed them. Its last miss was a fact in the caption under the picture; the third instructions said where to look ("in its picture and in the caption under it"), and it read all twelve right, in a median of 1.7 seconds. Without measuring, eleven out of twelve would have looked good enough.

A review by a session that didn't write the code

The examiner proves that the app does what the specification says. It can't find problems the specification doesn't mention. So one prompt starts a new session as an outside expert, with these orders: change no file, try to break the app, and prove each problem with a throwaway test before reporting it.

On the app rebuilt from our prompts, it reported 14 problems and proved 12 of them with throwaway tests. One unreadable file made the app throw away a whole upload of ten files. A 6400x6400 screen recording was accepted and sent the model a 179 MB request. The sign-up page told strangers which emails have accounts: 30 tries, all answered "already exists", none counted toward the limit.

The next prompt says: "If SPEC.md must change, tell me the change and wait for my answer." The session fixed five problems, writing a failing test first each time. On three it stopped and proposed the exact new wording for SPEC.md: a pixel limit for frames taken from recordings, counting refused sign-up tries toward the limit too, and "Python 3.14 or newer". We said yes.

One detail of its fix was better than ours. Once the sign-up limit is reached, its page shows an empty form. Our own version still showed the form as the person had filled it in, with "An account with this email already exists", which reveals exactly what the limit is there to hide.

What a real iPhone found

In our own build, the examiner passed everything. Then a photo picked in Safari on an iPhone was refused as "not a picture", three times, with nothing in the log.

We didn't guess. We added the log line that would have explained it: for each refused file, the type the browser declared, the size, and the first 16 bytes. The next try logged a JPEG of 1,609,318 bytes with a perfectly normal JPEG start. Safari sends an HDR photo as a JPEG holding a second picture, the HDR map, and Pillow calls that format MPO, not JPEG. One word added to the list of accepted formats, and the photo became a note in 15.8 seconds.

Breaking it on purpose

On a test bench (a virtual machine with Coolify, an iPhone over Tailscale, a fake mail server) we broke five things: the email server, the AI, the worker, a deploy, and the whole data volume. Three of the five changed the app:

  • With the mail server down, signing up made the account and then showed an error page, and the weekly email stopped at the first address that failed. Now the page says the email could not be sent, the log says why, and the weekly email continues to the next address.
  • A failure of the local model left no line in the log. Now it does, for every provider.
  • With the worker stopped, Coolify showed the app as unhealthy within a minute, but sent no email in the seven minutes before we started it again. The worker now records that it is alive every 30 seconds, and a check every ten minutes emails the operator once each time the worker goes silent.

The worst case was the cheapest to recover from: with the volume deleted, a fresh deploy, the backup copied back from another machine, and one restore command brought back every note, account and file in about a minute.

Facts your assistant may not know

An AI assistant only knows the world up to the date its training stopped. These would have broken the app, so we checked each one against the documentation and the installed package, not memory:

  • Claude's Python package sends the key as Authorization: Bearer, not the x-api-key header of the API's own examples.
  • Haiku 5.5 refuses a non-default temperature, and uses extended thinking by default, so the first block of its answer may be its thinking, not the text.
  • The package retries twice by default, so one save could take 360 seconds to read, against a 120-second limit.
  • Django 6.1 configures email through a MAILERS setting and refuses the older settings next to it.

Prompt 8.1 says: check every name you use against the anthropic package installed here, not your memory.

What we'd tell anyone building with an AI assistant

  1. Write down what "done" means before any code exists. Wherever you said nothing, the assistant makes a reasonable guess.
  2. Keep the tests that prove the app works where the assistant isn't allowed to change them, and let Git show that it didn't.
  3. Get a review from something that didn't write the code, and make it prove each finding.
  4. Let it stop and ask when the specification is wrong, and change the specification first.
  5. When something fails and the log is silent, add the line that would have explained it before you guess.

This comes from Ship an App by Directing AI, Volume 4 of Weekend AI Projects, a book series we publish on building working systems with an AI coding assistant. The free sampler has the opening pages and full contents of all five volumes: https://weekendaiprojects.gumroad.com/l/sampler. The book itself is at https://weekendaiprojects.gumroad.com/l/ship-an-app.

Top comments (0)