Instead of reading benchmark charts, I gave Claude Opus 5 an empty folder and one prompt: build a full-stack app. I didn't write any of the application code.
The result: a working project-management dashboard on Next.js + FastAPI + SQLite + JWT, with 64/64 backend tests passing. And a few honest gaps I'll show you too.
Why Opus 5
Anthropic released Opus 5 on July 24, 2026, positioned for complex agentic coding. It's the default model on Claude Max and the strongest on Claude Pro, so it's the one most people are actually using. Anthropic also claims it's much better at verifying its work and iterating until it succeeds. That's a claim I can test.
The rules
The project: FlowDesk, a production-style SaaS project manager.
- Stack: Next.js + TypeScript, FastAPI + Python, SQLite, JWT auth, a real test suite
-
Rules:
- Start from an empty folder (
challenge-app), no template - One master prompt
- I write no application code
- Claude fixes its own errors
- The app has to genuinely run, not just compile or look good in a screenshot
- Start from an empty folder (
It checked the machine first
Before writing code, it wrote the architecture down, then inspected the actual machine:
- ran
lsof, found port 3000 taken, and picked 3001 - noticed
uvwasn't installed and fell back to the standardvenv - even noted how little disk space was left
Most generated projects assume a clean machine and fall over on a real one. The final structure matched the plan it wrote at the start.
The backend
Not one giant main.py. Separate modules for config, database, models, schemas, security and dependencies, with one router per resource (auth, projects, tasks, users, dashboard). 18 endpoints across 11 routes, all typed and validated.
The most revealing moment: invented versions
It pinned exact dependency versions from memory, and three didn't exist:
bcrypt 5.0.1email-validator 2.4.0httpx 0.29.0
All plausible, all fictional, all just past the edge of its training data, where a model is least reliable. Its own diagnosis from the build log:
I invented a patch release. The newest bcrypt is 5.0.0, not 5.0.1.
It named the failure mode, fixed all three and carried on without being asked.
Lesson for your own projects: check AI-pinned versions against the real package index.
It also skipped passlib (the default in most FastAPI tutorials) because passlib reads a bcrypt attribute newer releases removed, and it wrote that reason in a comment.
The frontend
Next.js App Router, a shared API client, an auth context holding the token, and a route group that wraps every authenticated page in a guard, so no page has to remember to check. 16 components, with loading, empty and error states.
The seed data deserves a mention. The demo user is "Avery Chen, Director of Product Operations", and tasks look like "dunning email sequence for failed payments" and "fix IDOR on the export endpoint". A dashboard full of "Project One" reads like a tutorial; this reads like someone's real Tuesday.
It also checked the responsive layout itself at 1440, 768 and 375 px, measuring instead of assuming.
Did it run?
Yes, first try. Uvicorn on port 8000, Next.js ready in 330 ms, no traceback.
That's a less dramatic story but a better result: the failures had already happened during install and were fixed before anything started. I could have edited those errors in to look like a crash on first run. I didn't.
Without being told, it called its own health endpoint right after startup and got a 200. That's the difference between finishing and verifying.
The running app: projects with status, progress and due dates; tasks with priority, assignee and project, all persisted in SQLite. 7 projects, 40 tasks, 35% completion, 3 overdue, every number computed by the backend, not hardcoded.
Code review: what holds up
- Login doesn't leak accounts: an unknown user and a wrong password take the same path and return the same message
- Consistent errors: every error has the same shape, a code plus a message
What doesn't
- The token sits in localStorage, so route protection is client-side only
- No pagination anywhere
- The API can edit a task in five ways the UI never exposes
Verifying "done"
A model saying it's done and a model being done are different claims.
-
64/64 tests pass: auth, invalid login, protected endpoints, projects, tasks and profile. I read this off the raw
pytestoutput, not the model's summary. - Manual API checks:
- Protected endpoint, no token → 401
- Forged token → 401
- Correct password → real signed token
- Wrong password → 401
My scores
| Area | Score |
|---|---|
| Planning | 9/10 |
| Code quality | 8/10 |
| Interface | 9/10 |
| Debugging | 9/10 |
| Testing | 7/10 |
| Autonomy | 9/10 |
| Total | 51/60 |
Testing is the weakest: all 64 tests are backend-only, there are no frontend tests, and no production build was ever run. That's the gap a human still has to close before this ships.
Takeaways
- It planned, built, broke, diagnosed itself in plain language and got much further alone than I expected.
- Watch for invented package versions right past the training cutoff.
- "Tests pass" only covers what was tested. Check what the tests cover.
- Read the raw test output, not the model's summary of it.
Full build, start to finish: https://youtu.be/HMPVmqG1_lI
What should I ask an AI agent to build next? Tell me in the comments.
Drafted with AI assistance from the recorded build. I make AI, coding and tech tutorials on RainyTech.
Top comments (0)