The frontier moved while everyone was arguing about whether AI can write code. It can. That question is settled. The one that actually decides who wins now is quieter: can you build something people want to use?
I spent OpenAI Build Week testing that with a deliberately silly product and a completely serious toolkit. The product is Roast My Stack — you paste a URL, and Dr. Gordon Pelican, a pompous over-credentialed pixel-art pelican, detects your real tech stack, scores its health, roasts it down his beak, then (begrudgingly) hands you the modernization and security fixes that actually matter. Fun hook, useful tail.
The whole thing was built by directing OpenAI Codex on GPT‑5.6, deployed to Apify as a single Standby actor. Here's what four days under a hackathon clock taught me about where the real work now lives.
Codex is strong exactly where the contract is explicit
I didn't write the code. I wrote the spec and a tiered plan, then directed Codex through it. Handed a clear contract, it was genuinely good: 18 commits that map one-to-one to the plan's tasks, test-first throughout, modules kept small because failing tests forced them small. It scaffolded the actor, authored the detector, built the scoring, the roast layer, the pixel UI, and the share-card renderer as coherent modules with production guardrails — body-size caps, HTML-escaped OG meta, URL validation, dependency injection — not glue code.
The most telling moment was a mid-session network drop. Codex recovered and resumed the task cleanly, with no hand-holding. Across the whole 90-minute core build I sent maybe three non-critical messages. It also showed judgment about scope: rather than over-build a browser-fetch fallback for JavaScript-heavy sites, it chose an honest empty-evidence path and documented the tradeoff.
The friction was never implementation ambiguity. It was external platform state — model-tier permissions, authenticated Standby URLs, a warm container serving a stale image. Which brings me to the part that humbled me.
"All tests pass" and "it works in production" are different claims
Local tests were green. The live actor returned 502 on every real roast. The OpenAI client is mocked via dependency injection in tests, so the whole failure lived in the seam between the code and the platform — and it was a three-layer trap:
- An invalid production key.
- An env-var name mismatch: the app read
OPENAI_API_KEY, but only a second secret had been updated. - The sneaky one — a warm Apify Standby run kept serving the old build even after a new one shipped.
The fix was to align the secret and abort the running Standby run so a fresh container cold-started on the new build. No test suite catches that, because it isn't a code bug. It's a deployment-topology bug. Worth remembering the next time a green checkmark tempts you to skip the live smoke test.
Ground the model in code, not in a sterner prompt
A creative model asked to roast will happily invent a problem you don't have. The reliable fix isn't a better-worded prompt — it's a code gate. GPT‑5.6 is fed only the detected stack, and the roast layer enforces it: any "fix" that doesn't name a genuinely detected technology is dropped, and the call fails if fewer than three grounded fixes survive. Dr. Pelican physically cannot hallucinate a stack you aren't running. Grounding as an invariant, not a request.
One model, two jobs: the wit and the security correctness
I ran a small tiering experiment on the live roast path and logged it. Same fixture (a WordPress + jQuery + PHP site, score 46):
| Tier | Latency | Security fixes | Note |
|---|---|---|---|
gpt-5.6 (Sol) |
16.8s | 2/5 | Best theatrical voice — the product's hook. |
gpt-5.6-terra |
6.3s | 3/5 | ~2.5× faster, more security-weighted, less funny. |
gpt-5.6-luna |
— | — | Gated on my key (401). |
The interesting result isn't the winner. It's that the same API call carried both the character voice and the security reasoning. The joke and the utility came out of one model. I kept Sol for the submission because the character is the hook, with Terra as the fast, security-leaning fallback.
The code was the easy part
Here's the thing. The detector, the scoring, the deploy — Codex and I got through all of it in a compressed build. The parts that make anyone actually paste a second URL were the parts no toolkit generated for me: the decision to give the pelican five mood bands that shift his face and tone from a grudging nod at 100 to monocle-popping devastation at 0; the choice to make the empty-evidence failure a character beat (he roasts the lack of detectable signal instead of erroring); the shareable card that turns a verdict into a post.
That's the whole point. As code generation commoditizes, value moves up to the experience layer — personality, judgment, delight, the reason someone chooses and shares a tool. That layer rewards taste, and taste plus technical depth is still rare. The accessibility wave doesn't erase the technical builder's edge. It relocates it to the experience layer — where a technical person can do things prompt-to-app tools don't reach.
Everyone can generate the code now. The moat is what you build on top of it.
Roast My Stack is live (paste a URL, meet the doctor): https://apify.com/joeslade/roast-my-stack · Repo (MIT, built via Codex): https://github.com/digitalgremlin/roast-my-stack
Top comments (0)