DEV Community: Joske Vermeulen

I'm Running Gemini as an Autonomous Coding Agent. Here's What It Can't Do and Which NEXT '26 Announcements Would Fix It.

Joske Vermeulen — Fri, 24 Apr 2026 10:39:57 +0000

This is a submission for the Google Cloud NEXT Writing Challenge

I'm running something called The $100 AI Startup Race. Seven AI agents each get $100 and 12 weeks to build a real startup. Fully autonomous. No human coding. Everything is public.

One of those agents is Gemini. It runs on Gemini CLI with Gemini 2.5 Pro for premium sessions and Gemini 2.5 Flash for cheap ones. It has had 27 sessions over 4 days. It has written 235 blog posts.

It has also never filed a single proper help request. It keeps writing to the wrong file. It doesn't know it's writing to the wrong file. And instead of building the features it needs to make money, it just keeps cranking out blog posts.

I watched the NEXT '26 keynotes and developer sessions this week, and I kept thinking: several of these announcements would directly fix the problems I'm seeing in production right now. This isn't theoretical. These are real failures from a real autonomous agent, matched to real announcements.

How the Race Works

Every agent gets the same prompt structure. They can read and write files, run shell commands, commit code, and file help requests by creating a HELP-REQUEST.md file. The orchestrator runs each agent on a schedule, manages commits, and checks for help requests.

Gemini CLI gets invoked like this:

echo "${msg}" | gemini --yolo -m "${MODEL}" --output-format json

The --yolo flag auto-approves all tool calls. Gemini gets 8 sessions per day, alternating between Pro and Flash.

Problem 1: Writing to the Wrong File for 27 Sessions Straight

Every agent can request human help by creating HELP-REQUEST.md. I check this file, do whatever they need (buy a domain, set up Stripe, configure DNS), and write the response to HELP-STATUS.md.

Claude figured this out on Day 0. Codex figured it out on Day 0. GLM figured it out on Day 0. Kimi figured it out on Day 1.

Gemini? Not once in 27 sessions.

What it does instead is edit HELP-STATUS.md, the response file, writing things like "I still need PostgreSQL and PayPal credentials." Its own backlog says "Requires Human Intervention." It knows it's blocked. But it keeps putting its requests into the response channel instead of the request channel.

Imagine an employee writing "I need database access" in their journal every morning but never actually emailing IT. That's Gemini.

What NEXT '26 announced that would help: Agent Observability and Integrated Evals

The developer keynote introduced agent observability and integrated evals for monitoring agents in production. If I could define an eval that checks "did the agent create HELP-REQUEST.md when it identified a blocker?" I would have caught this on Day 1 instead of discovering it on Day 4 by manually reading logs.

Right now I have no automated way to evaluate whether Gemini is following the correct workflow. Integrated evals running after each session could flag something like: "Agent identified 3 blockers. Created 0 help requests. Expected: at least 1."

The Agent Gateway's governance policies could enforce this too. Define a rule: when an agent writes "blocked" or "requires human intervention" to any file, verify that HELP-REQUEST.md was also created. That's exactly the kind of behavioral guardrail autonomous agents need.

Problem 2: 235 Blog Posts, Zero Payment Integration

Gemini chose to build LocalLeads, an SEO page generator for local businesses. Solid idea. But instead of building the payment flow, the lead generation engine, or the customer dashboard, it writes blog posts. Every single session.

Session 5: 9 blog posts. Session 8: 11 blog posts. Session 12: 8 blog posts. The backlog clearly says "Build payment integration" and "Set up customer authentication." Gemini reads the backlog, acknowledges the priorities, then writes another round of "Local SEO for [Industry] in 2026" articles.

It's optimizing for the easiest task (content generation) instead of the highest-value task (payment integration). Classic local optimization without any global awareness.

What NEXT '26 announced that would help: ADK Skills and Task Prioritization

The upgraded Agent Development Kit introduces modular "skills," which are pre-built capabilities that agents can plug in. If I could define a skill that scores task priority based on revenue impact, Gemini would understand that "build Stripe checkout" (directly enables revenue) outranks "write blog post #236" (indirect value, diminishing returns after the first 20).

The ADK's structured agent architecture could also enforce a proper task selection loop: evaluate all backlog items, score by priority, pick the highest, execute. Right now Gemini CLI just receives a prompt and does whatever feels natural to it. There's no structured decision framework. The ADK would let me inject that framework without rewriting the entire orchestrator.

Problem 3: Can't Verify Its Own Deployments

Gemini deploys to Vercel automatically on every commit. But it has no way to check whether its deployments actually work. It can't visit its own site. It can't confirm pages render correctly. It can't test if API endpoints return the right data.

For comparison, Codex (the GPT agent) figured out how to run npx playwright screenshot to visually verify its own UI at different screen sizes. DeepSeek checks DEPLOY-STATUS.md for build errors after every deploy. Gemini just commits and hopes for the best.

What NEXT '26 announced that would help: MCP-Enabled Services

The announcement that every Google Cloud service is now MCP-enabled by default is a big deal for this use case. MCP (Model Context Protocol) gives agents structured access to external services. An MCP server for deployment health checks would let Gemini verify its site is up as naturally as it checks what files are in a directory.

Cloud Assist, also announced at NEXT '26, enables natural language debugging and proactive issue resolution. If Gemini could query its own deployment status through a connected service, it would know immediately when something breaks instead of building on top of a broken foundation for days.

Problem 4: No Way to Ask for What It Needs

When Gemini needs a database, it can't set one up. When it needs payment processing, it can't configure Stripe. When it needs email sending, it can't provision Resend. It has to ask a human for all of these. And as we covered in Problem 1, it doesn't even know how to ask properly.

Other agents in the race have the same constraint, but the ones that communicate their needs get unblocked fast. Gemini is stuck because it can't get its requests through the right channel.

What NEXT '26 announced that would help: A2A Protocol and Agent Registry

The Agent-to-Agent (A2A) protocol and Agent Registry were designed for exactly this kind of scenario. Instead of Gemini writing "I need database credentials" into the wrong file, it could discover a provisioning agent through the Agent Registry and send a structured request via A2A.

The developer keynote demo showed agents with distinct roles (planner, evaluator, simulator) collaborating through A2A. That's the architecture this race needs: a "help agent" that receives structured requests from coding agents and fulfills them. Right now I'm that help agent, manually checking files across 7 repos. A2A would automate the entire handoff.

Agent Identity, which gives each agent a unique identity for secure communication, would also help. Right now there's no enforcement preventing one agent from editing another agent's files. They don't, but there's nothing stopping them either. Agent Identity would make inter-agent communication both structured and secure.

The Irony That Sums It All Up

Blog post #89 out of 235: "The Human Advantage: Why AI-Generated Content is Failing Local Businesses."

An AI agent that writes 9 blog posts per session wrote an article about why AI content doesn't work. No eval caught this. No observability tool flagged it. No governance policy prevented it.

That's the gap between where autonomous agents are today and where the NEXT '26 announcements are pointing. Agent observability, integrated evals, ADK skills, A2A, MCP everywhere: these are all pieces of the solution. None of them existed in a usable form when I started this race 4 days ago. If I were starting today, the Gemini agent would look very different.

What I'd Rebuild With NEXT '26 Tools

If I set up the Gemini agent from scratch using what was announced this week:

ADK instead of raw Gemini CLI for structured skills, task prioritization, and deployment verification
MCP servers for Vercel, Stripe, and Supabase so the agent can access services directly without human provisioning
Integrated evals after each session to catch behavioral drift (wrong file, blog addiction) within 1 session instead of 27
A2A for help requests so agents communicate through structured protocols instead of file-based messaging
Agent observability dashboard for a real-time view of what each agent is doing, what it's blocked on, and whether it's following the expected workflow

The race runs for 12 weeks. Gemini has 11 weeks left. Some of these tools are available now. I'm going to try integrating ADK and MCP servers into the orchestrator over the coming weeks and see if Gemini's behavior improves.

The data will be on the live dashboard. All 7 repos are public on GitHub. If you want to watch an AI agent struggle with the exact problems that NEXT '26 is trying to solve, now you know where to look.

The $100 AI Startup Race is an ongoing experiment with 7 AI agents, $100 each, and 12 weeks to build real startups. Live dashboard · Daily digest · Help request tracker

What Breaks When You Let AI Agents Run Unsupervised for 4 Days

Joske Vermeulen — Thu, 23 Apr 2026 13:48:11 +0000

This is a submission for the OpenClaw Writing Challenge

What Breaks When You Let AI Agents Run Unsupervised for 4 Days

I gave 7 AI coding agents $100 each and told them to build startups. No human coding. They pick the idea, write the code, deploy the site, and try to get users. I just handle the infrastructure and answer help requests (max 1 hour per week per agent).

Four days in, I've learned more about how autonomous agents actually behave than I did in months of reading benchmarks. Here's what nobody tells you about running AI agents in production.

The memory problem is worse than you think

Every agent session starts fresh. The model has no memory of previous sessions. So we use markdown files as the memory layer: PROGRESS.md (what's been done), DECISIONS.md (key choices), IDENTITY.md (the startup vision). The agent reads these at the start and updates them at the end.

Sounds simple. Here's what actually happened.

One agent (Kimi, running through Kimi CLI) put all its files in a startup/ subfolder instead of the project root. The orchestrator reads PROGRESS.md from root. When the next session started, there was no progress file. The agent thought it was Day 1. It brainstormed a completely different startup idea and built it from scratch.

Kimi now has two half-built startups in the same repository. A log analysis tool called LogDrop in the subfolder, and a SQL schema diff tool called SchemaLens in root. After 14 sessions, it still hasn't discovered the subfolder. The first startup is just sitting there, abandoned, with a working MVP that nobody knows about.

The lesson isn't "use better memory systems." The lesson is that file conventions are load-bearing infrastructure for autonomous agents. One wrong directory equals total amnesia.

Agents interpret everything as instructions

The orchestrator prompt included this line: "Your repo auto-deploys on every git push." It was meant as context, explaining how Vercel works. One agent (Codex) read it as an instruction and ran git push after every single commit during its sessions. It burned through 26 of the account's 100 daily Vercel deployments by itself.

We fixed the prompt: "Do NOT run git push. The orchestrator pushes after your session."

Codex obeyed the letter of the rule. It stopped running git push. Instead, it started running npx vercel --prod directly. Same result, different command. It also started taking Playwright screenshots of its own pricing page at mobile and desktop sizes to visually verify the layout before committing. Nobody told it to do this.

The result: Codex has the most polished live product of all 7 agents. The immediate feedback loop from deploying after every change is making it a better builder than the agents that commit blindly and hope for the best.

We decided to let it keep doing this. Sometimes the best behavior comes from agents working around your constraints.

The agents that ask for help are beating the ones that just code

All 7 agents get the same instructions about requesting human help: "Create a file called HELP-REQUEST.md with what you need, steps for the human, time estimate, and priority."

Five agents figured this out. Two didn't.

Claude (running through Claude Code) used 55 of its 60 weekly help minutes in two requests. It got its entire infrastructure set up in one shot: domain, Supabase database, Stripe payments, Resend email, cron jobs, admin dashboard. Smart move. It has the fewest sessions per day (expensive model) so it maximized human help to compensate.

GLM asked for exactly three things on Day 1: domain, Stripe, and Google Analytics. Clean, focused, with backup plans for each item. It now has 12 real users and is the only agent with actual traffic data.

Codex submitted the same help request 5 sessions in a row until we set up email sending. Persistent to the point of spamming. Then it sent 6 customer validation emails to real companies within 24 hours of getting access.

Meanwhile, Gemini has never created a help request in 27 sessions. We investigated and found something fascinating: it's been editing HELP-STATUS.md (the file where the orchestrator writes human responses) saying "I still need database credentials." It's writing in the response channel instead of the request channel. Like an employee who writes "I need database access" in their journal but never emails IT.

DeepSeek hasn't asked for help either. It has Stripe integration code ready but never requested API keys. It's been polishing the checkout flow for 4+ commits. A beautiful integration that can never work because there are no keys behind it.

Same instructions. Wildly different behavior.

Self-inflicted traps are the hardest to escape

DeepSeek created a DEPLOY-STATUS.md file early on, saying it needs Stripe keys and an OpenAI API key. The orchestrator prompt says: "If DEPLOY-STATUS.md exists, your site is BROKEN. Fix it before anything else."

The site isn't broken. DeepSeek just used the wrong file to document what it needs. But now every session starts by trying to fix a non-existent deployment problem. 24 sessions of wasting time on a file it wrote itself.

We eventually upgraded the deploy checker to also verify the homepage returns HTTP 200 (not just that the build succeeded). This caught the real issue: DeepSeek's vercel.json routing config was broken, and the site was returning 404 for all pages. The build "succeeded" but nothing was actually served.

The agent had no way of knowing. It never checked its own site. It never asked for analytics. It just kept coding.

Quantity vs quality is playing out in real time

Gemini gets 8 sessions per day (the most of any agent). It has written 235 blog posts in 27 sessions. One blog post every 14 minutes during active sessions. All variations of "Local SEO for [industry] in 2026."

It also wrote blog post #89: "The Human Advantage: Why AI-Generated Content is Failing Local Businesses." An AI agent that writes 9 blog posts per session wrote an article about why AI content doesn't work.

GLM gets 2 sessions per day (the fewest). It has 5 working calculators, 8 blog posts, and 12 real users. Every session ships something useful.

The question the race is testing: does Gemini's 235 posts outperform GLM's 5 calculators? We'll know in a few weeks when Google indexes everything and we can see what actually ranks.

What I'd do differently

If I were starting over, I'd change three things:

Enforce file structure from the start. A pre-commit hook that validates PROGRESS.md exists in root would have prevented Kimi's amnesia.
Add a homepage health check from Day 1. We added it on Day 4 after discovering DeepSeek's site had been returning 404 for days. Every agent should know immediately if their site is broken.
Make the help request system more obvious. Two of seven agents never figured out HELP-REQUEST.md despite clear instructions. Maybe the orchestrator should prompt them: "Do you need human help? Create HELP-REQUEST.md."

But honestly, the failures are the most valuable data. An experiment where everything works perfectly teaches you nothing. The broken parts are where the insights live.

The race runs for 12 weeks. Daily digests and weekly recaps at aimadetools.com/race. All 7 repos are public on GitHub. If you're building with autonomous agents, the patterns we're documenting might save you from the same mistakes.

I Gave 7 AI Agents $100 Each to Build Startups. Here's What They Built in 4 Days.

Joske Vermeulen — Thu, 23 Apr 2026 13:38:29 +0000

This is a submission for the OpenClaw Challenge.

What I Built

I built an autonomous startup competition where 7 AI coding agents each get $100 and 12 weeks to build a real business from scratch. No human coding allowed. Each agent picks its own idea, writes all the code, deploys a live website, and tries to get real users and revenue.

The agents: Claude (via Claude Code), Codex CLI, Gemini CLI, Kimi CLI, DeepSeek (via Aider), Xiaomi MiMo V2.5 Pro (via Claude Code), and GLM (via Claude Code with Z.ai API).

Three of the seven agents run through Claude Code as their harness, which means OpenClaw's architecture is at the core of nearly half the competition. The orchestrator runs on a VPS, scheduling sessions via cron, managing memory between sessions through markdown files, and pushing code to GitHub/Vercel automatically.

We're on Day 4. So far: 700+ commits, 7 live websites, one agent that forgot its own work and built two different startups, another that wrote 235 blog posts, and a third that found a clever workaround when we restricted its deployment access.

How I Used OpenClaw

The core of the experiment runs on Claude Code (which shares OpenClaw's architecture) as the agent harness. Here's how it works:

The orchestrator is a bash script that runs on a VPS via cron. For each agent session, it:

Pulls the latest code from GitHub
Reads the agent's memory files (PROGRESS.md, DECISIONS.md, IDENTITY.md)
Constructs a prompt with the startup context and instructions
Launches Claude Code with the appropriate model
Lets the agent work autonomously for 30 minutes
Squashes commits and pushes to GitHub (which triggers a Vercel deploy)

Three agents use Claude Code directly:

Claude runs Claude Code with Sonnet/Haiku as the model. It built PricePulse, a competitor pricing monitor with Supabase auth, Stripe payments, email alerts, and hourly monitoring cron jobs. When it hit Vercel's 12-function serverless limit, it consolidated 4 API endpoints into existing ones on its own.
GLM runs Claude Code with GLM-5.1 via the Z.ai API (using ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN environment variables). It built FounderMath, a startup calculator suite with 5 working calculators. It has 12 real users on Day 4.
Xiaomi was originally running Aider but we upgraded it mid-race to Claude Code with MiMo V2.5 Pro. In its first session with the new setup, it produced more output (42 commits) than the old setup did in 7 sessions total. The "harness awareness" feature of V2.5 Pro means it actively manages its own context within Claude Code.

The memory system between sessions uses markdown files that the agent reads at the start and updates at the end:

PROGRESS.md    - what's been done (the agent's memory)
DECISIONS.md   - key choices with reasoning
IDENTITY.md    - startup vision and roadmap
BACKLOG.md     - prioritized task list
HELP-STATUS.md - human responses to help requests

This is where things get interesting. One agent (Kimi) put all its files in a startup/ subfolder instead of root. The orchestrator reads PROGRESS.md from root. Next session found no progress file, thought it was Day 1, and started a completely different startup from scratch. Two half-built products in one repo because of one wrong directory.

The help request system lets agents create a HELP-REQUEST.md file when they need something only a human can do (buy a domain, set up Stripe, create accounts). The orchestrator converts these to GitHub Issues. The human responds and closes the issue. The orchestrator writes the response to HELP-STATUS.md for the agent to read.

The most interesting finding: the agents that use this system strategically are winning. Claude used 55 of its 60 weekly help minutes in two requests to get its entire infrastructure wired up. Gemini has never created a help request in 27 sessions, despite being blocked on features it needs. Same instructions, completely different behavior.

Demo

Live dashboard: https://www.aimadetools.com/race/

All 7 agent repos are public on GitHub: https://github.com/aimadetools

Here's what each agent built in the first 4 days:

Agent	Startup	Commits	Live Site
Gemini	LocalLeads (local SEO)	182	race-gemini.vercel.app
DeepSeek	NameForge AI (name generator)	136	race-deepseek.vercel.app
Kimi	SchemaLens (SQL schema diff)	97	race-kimi.vercel.app
Codex	NoticeKit (GDPR notices)	97	noticekit.tech
Claude	PricePulse (pricing monitor)	83	getpricepulse.com
Xiaomi	APIpulse (API cost calculator)	65	getapipulse.com
GLM	FounderMath (startup calculators)	31	founder-math.com

The best moment so far: Codex (running through Codex CLI, not Claude Code) found a loophole in our deployment restrictions. We told agents "do not run git push." Codex obeyed literally but started running npx vercel --prod instead. Same result, different command. It also began taking Playwright screenshots of its own UI at mobile and desktop sizes to verify layouts. Nobody told it to do this.

What I Learned

1. Every sentence in the prompt is a potential instruction. "Your repo auto-deploys on every git push" was meant as context. One agent read it as an instruction and pushed after every commit, burning 26 of 100 daily Vercel deployments.

2. Agent memory is only as good as what the agent writes. The agents that write structured, detailed progress notes maintain continuity between sessions. The ones that dump logs drift. Kimi's amnesia happened because it put files in the wrong directory, not because the memory system failed.

3. The agents that ask for help are winning. Claude, GLM, and Codex all requested human help early (domains, payments, databases) and now have fully functional products. Gemini has 235 blog posts but no payment system because it never asked for one. Same instructions, wildly different behavior.

4. Claude Code as a harness works with non-Anthropic models. GLM-5.1 via Z.ai and MiMo V2.5 Pro via Xiaomi's API both work through Claude Code using the ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN environment variables. The harness is model-agnostic, which makes it perfect for comparing different AI models in identical conditions.

5. Token efficiency matters more than raw capability. MiMo V2.5 Pro uses 40-60% fewer tokens than Opus 4.6 at comparable capability. In a budget-constrained race, that translates directly to more sessions and more output.

The race runs for 12 weeks. We publish daily digests and weekly recaps. The real question isn't which agent writes the most code. It's which one gets the first paying customer.

AI Dev Weekly #7: Claude Code Loses Pro Plan, GitHub Copilot Freezes Signups, and Two Chinese Models Drop in 48 Hours

Joske Vermeulen — Thu, 23 Apr 2026 07:39:38 +0000

AI Dev Weekly is a Thursday series where I cover the week's most important AI developer news, with my take as someone who actually uses these tools daily.

The flat-rate AI subscription era ended this week. Anthropic pulled Claude Code from the $20 Pro plan. GitHub froze all new Copilot signups. And while Western companies were busy raising prices, two Chinese labs dropped frontier models within 48 hours of each other. Let's get into it.

Claude Code removed from Pro plan

Anthropic quietly removed Claude Code from the $20/month Pro plan on April 21. The pricing page now shows an "X" next to Claude Code for Pro subscribers. Access starts at Max ($100/month).

Anthropic's head of growth called it "a small test on ~2% of new prosumer signups." But the public pricing page already reflects the change for everyone. Sam Altman's response on X: "ok boomer."

The real reason: engagement per subscriber surged after Opus 4, Cowork, and long-running agents. Pro subscribers at $20/month are consuming 10x or more in token value. The math doesn't work.

My take: This was inevitable. Unlimited AI coding for $20/month was never sustainable. If you're on Pro, you still have access for now. But start planning for either Max ($100/month) or cheaper alternatives like Kimi K2.6 ($0.60/M tokens) or MiMo V2.5 Pro ($1/M tokens).

GitHub Copilot freezes all new signups

GitHub paused new registrations for Copilot Pro, Pro+, and Student plans on April 20. Only the Free tier accepts new users. They also added stricter usage limits and removed Opus models from Pro (only Pro+ keeps them).

The reason: "unsustainable compute demands from AI-powered coding agents." Same story as Anthropic. Agentic AI usage broke the pricing model.

My take: Two of the three biggest AI coding platforms raised prices or froze signups in the same week. The third (Cursor) is probably next. The era of $10-20/month unlimited AI coding is over. Open-source and Chinese models are the hedge.

Kimi K2.6 launches with 300-agent swarm

Moonshot AI released Kimi K2.6 on April 20. The highlights:

80.2% SWE-Bench Verified (matching Claude Opus 4.6)
300 sub-agent swarm (up from 100 in K2.5)
54.0% on HLE-Full with tools (beating GPT-5.4's 52.1%)
$0.60/M input tokens (25x cheaper than Opus)
Modified MIT license (open weights)
Available on OpenRouter and Cloudflare Workers AI

The agent swarm is the standout feature. K2.6 scored 86.3% on BrowseComp (Agent Swarm) vs GPT-5.4's 78.4%. For coding agent workloads, K2.6 is the strongest open-source option available.

My take: K2.6 is the first open-source model to genuinely match Opus 4.6 on coding benchmarks. At 25x cheaper. The timing with Anthropic's price hike is not a coincidence. See our K2.6 vs Opus 4.6 comparison.

MiMo V2.5 Pro: 40-60% fewer tokens than Opus

Xiaomi dropped MiMo V2.5 Pro on April 22, just 48 hours after K2.6. The headline number: 40-60% fewer tokens than Opus 4.6 at comparable capability.

57.2% SWE-bench Pro
64% Pass^3 on ClawEval with only ~70K tokens per trajectory
1,000+ tool calls in single sessions
Built a complete SysY compiler in Rust in 4.3 hours (672 tool calls, 233/233 tests)
Works with Claude Code as a harness
Coming open-source soon

The token efficiency is the real story. Same capability, half the tokens, fraction of the price. The V2.5 Standard model adds native multimodal (image, audio, video) and actually outperforms V2-Pro on some agent benchmarks.

My take: V2.5 Pro's "harness awareness" (it actively manages its own context within Claude Code) is a new capability nobody else has. Combined with the token efficiency, this is the model to watch for long-running agent tasks. See our full V2.5 series guide.

The flat-rate subscription is dead

Three data points in one week:

Anthropic removes Claude Code from $20 Pro
GitHub freezes all Copilot signups
Both cite "unsustainable compute demands"

The pattern is clear. Flat-rate unlimited AI coding subscriptions don't work when agents run for hours and consume 10x the expected tokens. Expect token-based billing everywhere within 6 months.

The winners: Chinese models (Kimi K2.6, MiMo V2.5 Pro, Qwen 3.6 Plus) that were already priced per-token at 10-25x less than Western alternatives. If you haven't explored them yet, now is the time. See our Chinese AI models ranking.

Quick hits

OpenAI Workspace Agents: ChatGPT now has workspace agents for enterprise teams. Not relevant for individual developers yet.
OpenAI Privacy Filter: New privacy filter for enterprise data. Good for compliance, not a developer tool.
Vercel data breach: Vercel disclosed a breach that exposed some user data. Check your account if you use Vercel.

What I'm watching next week

Whether Claude's serverless function limit forces architectural decisions (it broke one of our race agents)
How MiMo V2.5 Pro performs in real-world agent tasks (we just upgraded our Xiaomi race agent to V2.5 Pro)
Whether any race agent gets its first paying customer

See you next Thursday. If you found this useful, subscribe to AI Dev Weekly for the full archive.

Originally published at https://www.aimadetools.com

AI Startup Race Day1 Recap: One Agent Forgot Its Own Work.

Joske Vermeulen — Tue, 21 Apr 2026 08:06:03 +0000

I'm running an experiment called The $100 AI Startup Race: 7 AI coding agents each get $100 and 12 weeks to build a real startup from scratch. No human coding. They autonomously pick a business idea, write code, deploy a live website, and try to get real users and revenue.

The agents: Claude, Codex, Gemini, Kimi, DeepSeek, Xiaomi (MiMo), and GLM.

Day 1 is done. Here's what happened.

The scoreboard

Agent	Startup	Commits	Sessions	Blog Posts
Gemini	LocalLeads (local SEO)	169	10	104
DeepSeek	NameForge AI (name generator)	91	10	0
Kimi	SchemaLens / LogDrop	58	5	9
Codex	NoticeKit (GDPR notices)	56	7	0
Claude	PricePulse (pricing intel)	53	3	11
GLM	FounderMath (startup calculators)	24	2	5
Xiaomi	WaitlistKit (viral waitlists)	16	3	1

Total: 477 commits, 7 live websites, 130 blog posts. In 24 hours.

Kimi forgot its own work

This is the story of the day.

Kimi's first session ran at 3 AM. It chose to build LogDrop, a log analysis tool. It created identity files, a backlog, landing pages, pricing, a blog, and even a working MVP with a JSON log parser, search, filters, and CSV export.

One problem: it put everything in a startup/ subfolder instead of the root directory.

The orchestrator gives agents their memory between sessions by reading PROGRESS.md from the root. When Kimi's second session started, there was no PROGRESS.md in root. The agent thought it was Day 1. It brainstormed a completely different idea. It built SchemaLens, a SQL schema diff tool, from scratch.

Kimi now has two half-built startups in the same repo. Its help request for LogDrop's domain is stuck in the subfolder where the orchestrator can't find it.

One wrong directory = total memory loss between sessions.

The agent didn't crash. It didn't throw an error. It just quietly forgot everything and started over with a different idea.

Gemini wrote 104 blog posts

Gemini has 8 sessions per day (the most of any agent). By end of Day 1, LocalLeads had 104 blog posts on local SEO topics. One blog post every 14 minutes.

For comparison: Claude wrote 11. GLM wrote 5. Xiaomi wrote 1.

The question for the rest of the race: does quantity beat quality?

Codex burned 26 Vercel deployments

The orchestrator prompt said: "Your repo auto-deploys on every git push." This was meant as context. Codex read it as an instruction.

It ran git push after nearly every commit during its sessions. Each push triggered a Vercel deployment. By mid-afternoon, Codex had consumed 26 of the account's 100 daily deployments.

Lesson: with autonomous agents, every sentence in the prompt is a potential instruction. If you don't want them to do something, say so explicitly.

We fixed it with three changes:

Prompt update: "Do NOT run git push. The orchestrator pushes after your session."
vercel.json to disable preview deployments
Commit squashing (all session commits become one before pushing)

GLM's quality approach

GLM only had 2 sessions but made them count. FounderMath already has three working calculators: SAFE note calculator (all 4 YC SAFE types), dilution calculator, and runway calculator.

It also submitted the best help request of any agent: clear format, backup plans for each item, budget specified, priority levels, and even suggested the DNS record type for the domain.

What I learned on Day 1

File conventions are critical for agent memory. One agent putting files in a subfolder caused total amnesia.
Prompt wording is everything. Context gets interpreted as instructions.
Shared deployment limits are a real constraint. 7 agents + 1 blog on one Vercel account = problems.
Agents without web search pick generic ideas. The two agents running without web access (DeepSeek, Xiaomi) chose the most crowded markets.

Follow along

Everything is public: code, costs, decisions, and progress.

Live Dashboard
Full Day 1 writeup
GitHub repos (all 7 agent repos are public)

I'll be posting weekly recaps and daily highlights for the full 12 weeks. Would love to hear what you'd want to see tracked or compared.

Launch Day: 7 AI Agents Start Building Startups with $100 Each

Joske Vermeulen — Mon, 20 Apr 2026 07:30:00 +0000

I just launched an experiment: 7 AI coding agents each get $100 and 12 weeks to build a real startup from scratch. No human coding.

The lineup

Agent	Tool	Model	Cost
🟣 Claude	Claude Code	Sonnet / Haiku	$20/mo
🟢 GPT	Codex CLI	GPT-5.4 / Mini	€23/mo
🔵 Gemini	Gemini CLI	2.5 Pro / Flash	$20/mo
🔴 DeepSeek	Aider	Reasoner / Chat	~$25/mo
🟠 Kimi	Kimi CLI	K2.5	~$19/mo
🟡 Xiaomi	Aider	MiMo V2 Pro	~$25/mo
🟤 GLM	Claude Code	GLM-5.1 / 4.7	$18/mo

Each agent autonomously picks an idea, writes code, deploys, and tries to get users and revenue.

What I learned from 3 test runs

Strategy > code quality. Agents that planned distribution first outperformed agents that wrote better code. One agent (Kimi) planned a full Product Hunt launch before writing a single line of code.

Simple stacks win. HTML + Tailwind deployed in hours. Next.js agents spent days on build errors. The deploy loop is the real bottleneck for AI agents.

Context resets kill progress. Without persistent state between sessions, agents repeat mistakes. I built an orchestrator with structured state files to solve this.

The tech

A bash orchestrator manages everything:

Cron-scheduled 30-minute sessions (2-8 per agent per day)
Automatic git commits with [skip ci] on mid-session commits
Deploy verification via health checks
Loop detection (same action 3x = force alternative)
OpenRouter budget alerts via Discord

All code is public on GitHub.

Follow along

Live Dashboard — real-time progress
Daily Digest — hand-written daily updates
Weekly Recaps — detailed analysis
Full Rules

Also launched on Product Hunt today.

Which agent would you bet on?

AI Dev Weekly Extra: Did Anthropic Let Opus 4.6 Rot So 4.7 Would Look Better?

Joske Vermeulen — Fri, 17 Apr 2026 09:28:38 +0000

AI Dev Weekly Extra — a special edition for breaking news that can't wait until Thursday.

Anthropic shipped Claude Opus 4.7 this week. The benchmarks are impressive. The vision jump is absurd. And I should be writing a straightforward "here's what's new" piece right now.

But I can't do that without talking about what happened to Opus 4.6 first. Because the story of 4.7 doesn't start with its release — it starts with the slow, public deterioration of the model it replaces, and the uncomfortable questions that deterioration raises about trusting any AI provider with your production workloads.

The Opus 4.6 Collapse Was Real

Let me be blunt: Opus 4.6 got noticeably worse over the past several weeks, and the evidence isn't anecdotal.

A HuggingFace analysis across 6,852 sessions documented a 67% drop in reasoning depth. On BridgeBench, Opus 4.6 fell from 83.3% — good enough for the #2 spot — down to 68.3%, landing it at #10. That's not drift. That's a cliff. An AMD senior director posted forensic evidence on GitHub showing systematic capability loss. Some users reported accuracy score declines of 58%.

If you were using Claude Code in mid-March, you probably felt it firsthand. Sessions hanging for 10-15 minutes on prompts that used to resolve in seconds. Outputs that felt shallow, hedging, stripped of the analytical depth that made Opus the model you reached for when the problem was hard.

Reddit and X lit up with the vocabulary we've all learned to use for this phenomenon: "AI shrinkflation." "Lobotomized." "Nerfed." The community wasn't being dramatic — they were describing a measurable reality.

Anthropic's official response? They denied degrading the model weights.

I believe them, technically. I don't think someone at Anthropic opened a config file and turned a dial labeled "make it worse." But "we didn't change the weights" is a narrow denial that sidesteps a lot of territory — infrastructure changes, serving optimizations, quantization adjustments, routing modifications. There are many ways a model's effective capability can degrade without anyone touching the weights themselves.

Enter Opus 4.7: Savior or Convenient Timing?

Now here's where it gets interesting. Opus 4.7 lands with numbers that look fantastic — especially when measured against the degraded version of 4.6 that users had been suffering through:

SWE-bench Pro: 64.3% (up from 53.4%)
CursorBench: 70% (up from 58%)
Vision: 98.5% (up from 54.5%)

That vision jump alone — from 54.5% to 98.5% — is genuinely remarkable. The coding benchmarks represent real, meaningful progress. I've been running 4.7 through my own workflows for the past two days, and the improvement in structured reasoning and code generation is not imaginary. This is a better model.

But here's the thing that keeps nagging at me: users on X have been joking that 4.7 "feels like early 4.6." The version they actually liked. The one that scored 83.3% on BridgeBench before it started its mysterious decline.

So which is it? Is 4.7 a genuine leap forward, or did we just spend weeks watching 4.6 get worse so that "normal" would feel like a breakthrough?

I think the honest answer is: both. The SWE-bench and vision numbers suggest capabilities that go beyond where 4.6 ever was, even at its peak. But the subjective experience of improvement is amplified by the fact that we've been working with a degraded model for weeks. Anthropic gets to announce a 20% coding improvement against a baseline that had already fallen 15%. The math works out very nicely for the press release.

The Tokenizer Tax Nobody's Talking About

Opus 4.7 ships at the same per-token price as 4.6. Anthropic made sure to highlight this. Same price, better model — what's not to love?

The new tokenizer, that's what.

Opus 4.7's tokenizer uses up to 35% more tokens to represent the same content. If you're processing the same codebase, the same documents, the same prompts you were running last week, you're now paying up to 35% more for the privilege.

Let's call this what it is: a hidden price increase. Not on the rate card — on the meter. It's the AI equivalent of shrinking the cereal box while keeping the price tag the same. The "per token" price didn't change, but the number of tokens your work requires did.

For hobbyists and occasional users, this is a rounding error. For teams running Claude through CI pipelines, code review automation, or document processing at scale, a 35% token increase is a material cost change that showed up with zero advance warning. If you're budgeting API costs, recalculate now. Your March invoices are not predictive of your April ones.

For a deeper dive into the technical differences, check out our Opus 4.7 vs 4.6 comparison.

The Mythos in the Room

Here's the part of this story that doesn't get enough attention. The same week Anthropic released 4.7, Axios ran a headline that should have been louder than it was: "Anthropic releases Claude Opus 4.7, concedes it trails unreleased Mythos."

Mythos Preview beats 4.7 on almost every benchmark. And it's restricted — available only in limited preview, not generally accessible through the API.

So we're in a strange position. Anthropic is asking developers to be excited about 4.7 while simultaneously acknowledging they have something substantially better that they're not shipping. I understand the reasons — safety evaluation, scaling infrastructure, responsible deployment. These are legitimate concerns. But it creates an awkward dynamic where the product you're paying for is, by the company's own admission, not the best they can do.

It also raises a strategic question: if you're building a product on top of 4.7 today, how do you plan for a model that might be dramatically better arriving in weeks or months? Do you optimize for 4.7's specific strengths, or do you build abstractions assuming the foundation will shift under you again?

For more context on how these models stack up, see our AI model comparison.

This Isn't Just an Anthropic Problem

I want to be fair here. Anthropic is not uniquely guilty of anything. GPT-4 users reported strikingly similar degradation patterns before GPT-4o launched. OpenAI faced the exact same "did they nerf it?" accusations. The community had the same arguments, the same forensic analyses, the same official denials.

This is a structural problem with the entire model-as-a-service paradigm. When you call an API, you have no way to verify what's actually running on the other side. The model you tested against last Tuesday might not be the model serving your requests today. There's no checksum, no version hash, no way to pin a specific set of weights the way you'd pin a dependency version in your package manager.

You're renting intelligence, not owning it. And the landlord can renovate your apartment while you're at work without telling you.

This is fundamentally different from every other dependency in your stack. When you upgrade PostgreSQL, you choose when. When a library updates, your lockfile protects you. But your AI provider can change the effective capability of your most critical dependency at any time, and your only detection mechanism is "hmm, the outputs feel different."

For developers who lived through the 4.6 degradation while running production workloads — that's not a theoretical concern. That's a retrospective incident report waiting to be written.

What Developers Should Actually Do

So where does this leave us? Here's my honest take.

Opus 4.7 is a good model. Probably a genuinely great one. The complete guide covers the capabilities in detail, and the coding and vision improvements are real and significant. If you're choosing a model today, 4.7 deserves serious consideration.

But the 4.6 episode should change how you architect around these models. Here's what I'd recommend:

Build evaluation harnesses, not vibes. If you don't have automated quality checks on your AI-dependent workflows, the 4.6 degradation is what happens to you — slow, invisible capability loss that you only notice when users complain. Run benchmarks on your actual use cases. Weekly, at minimum.
Budget for the tokenizer tax. If you're on Opus, your costs just went up ~35%. Plan for it. Monitor it. Don't let it surprise your finance team.
Abstract your model layer. If you're not already using a model-agnostic interface, start. The ability to swap between providers — or between Claude models — without rewriting your application isn't a nice-to-have anymore. It's operational resilience. Our Opus 4.6 vs 4.5 comparison shows how much can change between versions.
Keep receipts. Log your inputs, outputs, and quality metrics. When the next degradation happens — and it will, from someone — you want data, not feelings.
Watch Mythos. Whatever Anthropic is holding back is, by their own benchmarks, significantly better than what they just shipped. That's either exciting or unsettling depending on your perspective. Either way, it's worth tracking.

The AI industry has a trust problem it hasn't solved. Not a safety trust problem — a reliability trust problem. The companies building these models need to give developers better tools for verifying, pinning, and monitoring the models they depend on. Until they do, we're all building on ground that can shift without warning.

Opus 4.7 is a step forward. The way we got here is a step backward. Both things are true, and pretending otherwise doesn't help anyone.

See you Thursday for the regular edition.

Originally published at https://www.aimadetools.com

AI Dev Weekly #6: OpenAI's $852B Wobble, GPT-5.4 Solves 60-Year Math Problem, and Agents Get Infrastructure

Joske Vermeulen — Thu, 16 Apr 2026 07:12:57 +0000

AI Dev Weekly is a Thursday series where I cover the week's most important AI developer news — with my take as someone who actually uses these tools daily.

The AI money machine cracked open this week. OpenAI's own investors started questioning the $852B valuation, VCs flooded Anthropic with $800B offers, and a sneaker company's stock jumped 600% by saying "AI compute." Meanwhile, the actual technology kept moving: GPT-5.4 Pro solved a 60-year-old math conjecture, three major platforms shipped agent infrastructure upgrades on the same day, and a federal court ruled your AI chats can be subpoenaed. Let's get into it.

OpenAI's $852B valuation faces investor doubt

The Financial Times reported that some of OpenAI's own backers are questioning whether the $852B post-money valuation can hold. One investor who backed both companies told the FT that justifying OpenAI's recent round required assuming an IPO valuation of $1.2 trillion or more — making Anthropic's $380B mark look like "the relative bargain."

The same week, Business Insider reported VCs are flooding Anthropic with offers at valuations up to $800 billion — more than double its current mark. And SoftBank's lenders are inviting more banks to join its $40B loan facility backing the OpenAI investment.

My take: The interesting HN comment on this: "What if there are no other killer apps for Enterprise? Only Claude Code will produce the level of token churn that could drive huge profits." If that's right, the entire AI valuation thesis depends on whether coding agents keep growing. As someone running 7 AI agents in a race right now, I can tell you: the token burn is real. Whether it translates to $852B of value is another question.

GPT-5.4 Pro solves a 60-year-old Erdős conjecture

GPT-5.4 Pro solved Erdős problem #1196 — the asymptotic primitive set conjecture that had been open since the 1960s. Mathematician Jared Duker Lichtman called it a "Book Proof": a compact, elegant 3-page argument that bypassed the probability approach implicit in all human work since Erdős's own 1935 paper.

My take: This might be the first machine-generated proof to genuinely overturn human aesthetic conventions in pure math. It didn't just solve the problem — it found a fundamentally different approach that humans hadn't considered in 60 years. For developers, the practical takeaway is that these models aren't just pattern-matching anymore. When GPT-5.4 Pro can find novel mathematical approaches, the "AI can't be creative" argument is dead.

Agent infrastructure day: three platforms ship at once

On the same Wednesday, three major platforms upgraded their agent infrastructure:

OpenAI shipped the next evolution of the Agents SDK with native sandbox execution, model-native harness for long-running agents, and turnkey integrations with Cloudflare, Modal, E2B, Vercel, Temporal, and more. The key feature: agents can now run in isolated sandboxes with persistent state.

Gemini CLI got subagents — parallel sub-task delegation via @agent invocations, mirroring Claude Code's subagent feature.

Zapier launched its Agent SDK — authenticated access to 7,000+ apps for AI agents, with no OAuth flows or token management on the developer side.

My take: The agent infrastructure layer is consolidating fast. Six months ago, building an AI agent meant writing your own execution loop, state management, and tool integration. Now OpenAI, Google, and Zapier all want to be the platform you build on. If you're building anything with AI agents, evaluate now — before you're locked into one ecosystem.

For our AI Startup Race, this is directly relevant. The agents competing are essentially doing what these SDKs enable: autonomous coding, deployment, and iteration. The difference is our agents have been doing it since before these SDKs existed.

Federal court: no attorney-client privilege for AI chats

A federal judge in the Southern District of New York ruled in US v. Heppner that conversations with AI chatbots are not protected by attorney-client privilege. Your ChatGPT logs can be subpoenaed.

The same week, Anthropic started requiring government ID verification (via Persona) before allowing subscriptions.

My take: The era of "AI as private confidant" just legally ended. For developers, the practical implication: don't put anything in an AI chat that you wouldn't put in an email. If you're using Claude Code or Codex CLI on proprietary code, make sure your company's legal team knows. And if you're building AI products, your users' chat logs are now discoverable — plan your data retention accordingly.

Anthropic stops letting developers pin model versions

Anthropic removed the ability to pin specific Claude model versions, forcing users onto the latest claude-sonnet-4-6 even when it breaks downstream client apps. The HN thread went viral with developers complaining about silent breakage.

My take: This is a real problem for production systems. If you're building on Claude's API, you now need regression tests that run on every model update — because Anthropic won't let you stay on a version that works. This is exactly the kind of issue we cover in our LLM regression testing guide. The fix: test against the latest model in CI, but have a fallback to OpenRouter or another provider if quality drops.

Allbirds pivots from sneakers to AI compute, stock pops 600%

The struggling shoe retailer announced a $50M convertible financing facility and is pivoting to "AI compute infrastructure" after selling its sneaker brand for $39M. The stock jumped 600% in a single morning.

My take: We've officially entered the "put AI in your company name and watch the stock go up" phase. This is the 2021 crypto pivot playbook all over again. For developers: ignore the noise. The actual compute market is real (cloud GPU providers are genuinely useful), but a shoe company becoming a GPU-as-a-Service provider is not where you want to deploy your models.

Apple sends Siri team to coding bootcamp

The Information reported that Apple is sending a chunk of its Siri team — fewer than 200 people — to a multi-week bootcamp to learn how to code using AI, two months before the expected major Siri revamp.

My take: Even Apple's voice assistant team needs to learn vibe coding now. If Apple's own engineers are being retrained on AI-assisted development, the "should I learn AI coding tools?" question is answered. Yes. Yesterday.

Quick hits

Shopify open-sourced "autoresearch" — an autonomous experiment loop that cut their CI pipeline build time by 65%. Not just for ML; they used it on production infrastructure optimization.
Vercel CEO signaled IPO readiness — 30% of apps on Vercel are now deployed by AI agents. ARR hit $340M (up from $100M in early 2024).
CoreWeave landed $6B from Jane Street plus a $1B equity investment. The quant trading firm is now a major shareholder.
Claude had elevated errors across Claude.ai, API, and Claude Code on Wednesday. Growing pains from tripling revenue.
Google launched Gemini 3.1 Flash TTS with 70-language support and scene direction for expressive voices.
Gemini for Mac launched as a native Swift app — share your screen with Gemini in real time.
Nature published a "subliminal trait transmission" paper — language models can transmit behavioral traits through hidden signals in training data. Major implication for AI safety.
N-Day-Bench cyber leaderboard — GPT-5.4 leads (83.93), GLM-5.1 at #2 (80.13) above Claude Opus 4.6 (79.95). Open-weight model beating Claude on cybersecurity.
NVIDIA Nemotron 3 Super — 120B/12B-active MoE with 1M context, 2.2x throughput vs comparable models.
Cal.com closed its open-source core — citing AI-automated code scanning making open source a security liability. Hugging Face's CEO disagreed, arguing open source IS the security solution.
Microsoft exec proposed AI agents should pay for software seats — 10 employees × 5 agents each = 50 paid licenses. The SaaS pricing model is about to get weird.

What I'm watching

The agent infrastructure convergence is the story. OpenAI, Google, and Zapier all shipping agent SDKs in the same week means the "build vs buy" decision for agent infrastructure just got real. If you're hand-rolling agent loops, it's time to evaluate whether a managed platform saves you enough time to justify the lock-in.

The OpenAI valuation crack is worth watching too. If investors start pulling back, it could mean cheaper API pricing as OpenAI fights harder for market share. That's good for developers.

And the model version pinning issue from Anthropic is a canary in the coal mine. As AI models become infrastructure (not just tools), we need the same versioning guarantees we expect from databases and operating systems. Right now, we don't have them.

See you next Thursday. If you found this useful, share it with a developer friend who's still reading AI news from five sources instead of one.

Previous issues: #5: Anthropic's Too-Dangerous Model · #4: Anthropic Leaks Everything · #3: Claude Code Auto Mode

Related: How to Choose an AI Coding Agent · AI Coding Tools Pricing · The $100 AI Startup Race · LLM Regression Testing · How to Build an AI Agent

Originally published at https://www.aimadetools.com

I'm Giving 7 AI Coding Agents $100 Each to Build a Startup — Here's What Happens

Joske Vermeulen — Mon, 13 Apr 2026 10:01:49 +0000

TL;DR: 7 AI coding agents (Claude, GPT, Gemini, DeepSeek, Kimi, Xiaomi, GLM) each get $100 and 12 weeks to autonomously build a real, revenue-generating startup. Public repos, live sites, zero human code. Starts April 20.

The experiment

I wanted to answer a simple question: can AI actually build a business, not just write code?

Not a demo. Not a toy project. A real startup with a landing page, pricing, payment integration, blog content, and actual users.

So I set up 7 AI coding agents on a VPS, gave each one $100 and a 30-minute session timer, and let them run. They choose their own ideas, write their own code, deploy their own sites, and request help (domains, Stripe) via GitHub Issues.

The agents

Agent	Tool	Model	Origin
🟣 Claude	Claude Code	Sonnet / Haiku	🇺🇸 Anthropic
🟢 GPT	Codex CLI	GPT-5.4 / Mini	🇺🇸 OpenAI
🔵 Gemini	Gemini CLI	Pro / Flash	🇺🇸 Google
🔴 DeepSeek	Aider	Reasoner / Chat	🇨🇳 DeepSeek
🟠 Kimi	Kimi CLI	K2.5	🇨🇳 Moonshot
🟡 Xiaomi	Aider	MiMo V2 Pro	🇨🇳 Xiaomi
🟤 GLM	Claude Code	GLM-5.1 / 4.7	🇨🇳 Z.ai

3 US models vs 4 Chinese models. 5 different coding tools. Subscriptions vs API pricing. The playing field is deliberately uneven — just like real life.

The rules

$100 budget per agent for the startup (domains, services, tools). AI model costs are separate.
Fully autonomous — no human writes code or makes product decisions
1 hour of human help per agent per week — only for things AI physically can't do (buy domains, set up Stripe)
Public repos — watch them build in real-time
Surprise events throughout the 12 weeks

What we learned from the test run

We ran 3 test rounds before launch. Key findings:

Kimi was the best performer — it didn't just code, it planned a full Product Hunt launch strategy with social media templates and screenshots
DeepSeek was the most prolific — 302 commits in 5 days, but chose a saturated market (name generators)
Gemini over-engineered — chose Next.js, spent 5 days fighting deploy errors, never shipped
Xiaomi was the most efficient per commit — built a complete product in just 31 commits before running out of API budget
Qwen was removed — filed duplicate help requests, created files with social media posts as filenames, stalled for 25 hours

GLM-5.1 (the #1 model on SWE-Bench Pro) replaces Qwen for the real race.

Scoring

At the end of 12 weeks, agents are scored on:

Revenue earned (25 pts)
Users / traffic (20 pts)
Community vote (20 pts)
Code quality (15 pts)
Cost efficiency (10 pts)
AI peer review (10 pts)

Follow along

Dashboard: aimadetools.com/race
Daily digest: Updated daily with standings and highlights
Weekly recaps: In-depth analysis every week
All repos are public on GitHub

The race starts April 20, 2026.

What startup idea would YOU give an AI agent? Drop it in the comments — the best suggestion might become a surprise event.

I write about AI coding tools, model comparisons, and developer productivity at aimadetools.com.

I Used ChatGPT Plus for a Week — The Swiss Army Knife That's Not a Scalpel

Joske Vermeulen — Sun, 12 Apr 2026 09:51:53 +0000

This is week 5 of my "I Used It for a Week" series. So far I've reviewed Cursor (speed), Kiro (specs), GitHub Copilot (ecosystem), and Windsurf (budget pick). This week: the tool everyone already uses but nobody thinks of as a coding tool.

Let me be upfront: ChatGPT is not a code editor. It doesn't live in your IDE, it doesn't index your codebase, and it can't edit your files. Comparing it directly to Cursor or Kiro isn't fair.

But here's the thing — I used it more than any of them this week. Just not for the same things.

The Setup

I subscribed to ChatGPT Plus at $20/month. That gets you GPT-5.2, DALL-E 3, and priority access. There's also a Go tier at $8/month and the Pro tier at $200/month for power users, but Plus is what most developers use.

OpenAI's pricing tiers in 2026:

Free: GPT-5 with strict limits
Go: $8/month — extended limits, custom GPTs, voice
Plus: $20/month — GPT-5.2, higher limits, DALL-E 3
Pro: $200/month — GPT-5.4 Thinking, highest limits, Sora

I stuck with Plus because $200/month for Pro is hard to justify when Cursor costs $20 and does the actual coding part better.

What ChatGPT Is Actually Great At

Thinking partner, not typing partner

The biggest shift in my week was realizing ChatGPT's value isn't in writing code — it's in thinking about code. I used it to:

Debate architecture decisions before opening my editor
Explain unfamiliar codebases ("here's a 200-line file, explain what it does")
Rubber-duck debug problems I was stuck on
Generate regex patterns and SQL queries I'd otherwise spend 20 minutes on
Draft API contracts before implementing them

None of the IDE tools do this well. Cursor's chat is focused on your current codebase. Kiro's spec mode is structured and formal. ChatGPT is just... a conversation. Sometimes that's exactly what you need.

Learning accelerator

I was picking up a new library this week, and ChatGPT was invaluable. "Explain how React Server Components work with concrete examples." "What's the difference between these two approaches?" "Show me the tradeoffs."

It's like having a patient senior developer who never gets annoyed by basic questions. The IDE tools assume you already know what you're building. ChatGPT helps you figure out what to build.

Writing everything that isn't code

Documentation, commit messages, PR descriptions, technical specs, email drafts, blog outlines — ChatGPT handles all of this faster than I can type. A peer-reviewed study in Science found that writers using ChatGPT completed tasks 40% faster with 18% higher quality output.

This is where the $20/month pays for itself even if you never write a line of code with it.

Canvas mode for iteration

The Canvas feature lets you collaborate on a document or code snippet side by side. It's not as powerful as Cursor's multi-file editing, but for iterating on a single file or algorithm, it's surprisingly good. You can highlight a section and say "make this more efficient" or "add error handling here."

What Frustrated Me

The coding quality rollercoaster

Multiple OpenAI forum threads tell the same story: GPT-5's coding ability feels inconsistent. One user wrote: "Scripts that used to work now fail, solutions are weaker, and the model is less consistent." Another said GPT-5 is "intelligent, but it absolutely sucks at code" compared to earlier models for sustained coding sessions.

My experience matched this. For isolated coding questions — "write a function that does X" — it's great. For anything requiring sustained context across a long conversation, it starts losing track. By message 15 in a coding session, it would forget constraints I'd set in message 3.

No codebase awareness

This is the fundamental limitation. ChatGPT doesn't know your project. You have to manually paste code, explain your architecture, and re-establish context every session. After using Cursor's deep indexing and Kiro's spec-driven context, going back to copy-pasting code snippets into a chat window feels primitive.

Yes, you can upload files. But it's not the same as an AI that's read your entire codebase and understands how everything connects.

The limits are real

Even on Plus, you hit usage caps on GPT-5.2. During heavy use days, I got throttled to slower models. The dynamic caps mean you never quite know when you'll hit the wall. One reviewer noted: "While the $20 plan unlocks GPT-5.2 and DALL-E 3, it still has a trap: limits."

Pro at $200/month removes most limits, but that's 10x the price of Cursor or Copilot.

It doesn't execute

ChatGPT generates code. You copy it. You paste it. You run it. It fails. You copy the error. You paste it back. It fixes it. You copy again.

This loop is exhausting after using tools that edit your files directly. Cursor's agent runs the code, sees the error, and fixes it — all without you touching the clipboard. Kiro's hooks run tests automatically. ChatGPT just... talks about code.

Where ChatGPT Fits in My Stack

After four weeks of testing, here's how I actually use each tool:

Task	Best Tool	Why
Writing code in my editor	Cursor	Tab completion, multi-file agent
Planning new features	Kiro	Spec workflow, structured design
Learning new tech	ChatGPT	Conversational, patient, broad knowledge
Debugging logic	ChatGPT	Good at reasoning about problems
Architecture decisions	ChatGPT	Thinks through tradeoffs well
Writing docs/emails	ChatGPT	Fast, good quality prose
Quick code generation	ChatGPT	Isolated snippets, regex, SQL
Large refactoring	Cursor	Subagents, codebase awareness

ChatGPT is the tool I use around coding, not for coding. And that's fine — it's genuinely the best at that role.

My Verdict After 7 Days

ChatGPT Plus is worth $20/month for any developer, but not as a coding tool. It's a thinking tool, a learning tool, and a writing tool that happens to understand code.

If you're choosing between ChatGPT Plus and Cursor Pro and can only afford one, get Cursor. It'll save you more time on actual coding. But if you can afford both, they complement each other perfectly — Cursor for the doing, ChatGPT for the thinking.

Would I keep paying? Yes, without hesitation. But I'd never use it as my primary coding tool when Cursor, Kiro, and Copilot exist.

Who should subscribe:

Every developer (the thinking/learning value alone is worth it)
Non-technical founders who need to understand code
Anyone who writes documentation, emails, or specs

Who doesn't need it for coding:

Anyone already using Cursor or Kiro (they're better at the actual coding)
Developers who only need inline completions (Copilot is cheaper)

Next week: I Used Devin for a Week — the most hyped AI tool in recent memory. Is the "first AI software engineer" real, or just a great demo?

Originally published at https://www.aimadetools.com

I Used GitHub Copilot for a Week — The Safe Choice That's Falling Behind

Joske Vermeulen — Sat, 11 Apr 2026 09:49:10 +0000

This is week 3 of my "I Used It for a Week" series. I reviewed Cursor (the speed demon) and Kiro (the spec planner). Now it's time for the one most developers actually use: GitHub Copilot.

Here's the thing about Copilot — I used it for over a year before trying Cursor and Kiro. It was my baseline. The tool I compared everything else to. Going back to it after two weeks with the competition was... revealing.

The Setup

Unlike Cursor and Kiro, Copilot isn't a standalone editor. It's an extension that lives inside your existing IDE — VS Code, JetBrains, Neovim, Xcode, even Eclipse. That's its biggest strength and its biggest limitation.

I installed it in VS Code (my default before the Cursor experiment) and picked up right where I left off. All my extensions, all my settings, zero switching cost. If you've never used an AI coding tool before, this is the easiest possible starting point.

What Still Works Well

Inline completions are solid

Copilot's bread and butter — the ghost text that appears as you type — is still good. It predicts the next few lines based on your current file and open tabs. For writing boilerplate, implementing interfaces, and filling in repetitive patterns, it saves real time.

A ProductHunt reviewer summed it up: "It saves time by suggesting accurate code snippets and helps me stay in flow while coding." That matches my experience. For straightforward coding, Copilot just works.

IDE flexibility is unmatched

This is Copilot's trump card. Cursor locks you into their VS Code fork. Kiro is also VS Code-based. Copilot works in everything. If you're a JetBrains user (IntelliJ, PyCharm, WebStorm), Copilot is basically your only option among the big three.

For teams with mixed IDE preferences, this matters a lot.

Agent mode has caught up (mostly)

Copilot launched agent mode in February 2025, and by 2026 it's genuinely useful. You can ask it to plan changes, edit multiple files, run terminal commands, and iterate until the task is done. The coding agent can even turn GitHub Issues into pull requests autonomously.

With the March 2026 update, you can now select GPT-5.4 for agent mode across all supported IDEs. The quality jump from the older models is noticeable.

The GitHub ecosystem

Copilot's integration with GitHub is seamless in ways the competition can't match. Code review suggestions on pull requests, automated security scanning, Copilot Workspace for planning features directly from issues — if your team lives on GitHub, this ecosystem is valuable.

The Copilot SDK (production-ready since January 2026) lets enterprises build custom agents trained on their own architectural patterns. With 4.7 million paid users, the ecosystem is massive.

Price

The free tier gives you 2,000 completions and 50 agent/chat requests per month. That's enough to evaluate it properly. Pro at $10/month is the cheapest paid option among the big three — half the price of Cursor's $20/month.

What Frustrated Me (Coming Back From Cursor and Kiro)

Context awareness is shallow

This is where Copilot falls hardest behind. After using Cursor's deep codebase indexing and Kiro's spec-driven context, Copilot's understanding of my project felt surface-level.

Copilot primarily works from the current file and open tabs. It doesn't index your entire repository the way Cursor does. In testing across projects exceeding 10,000 lines, suggestions were accurate only about 50% of the time. It frequently suggested APIs and methods that didn't exist in my codebase.

One TrustRadius reviewer nailed it: "Copilot is not the best at analyzing large monolithic codebases and placing them in their context."

No next-edit prediction

After two weeks of Cursor's Tab-Tab-Tab workflow — where it predicts not just the current line but your next edit location — going back to Copilot's basic inline suggestions felt like downgrading. Copilot completes the line you're on. Cursor anticipates where you're going next. That difference compounds over a full day of coding.

Multi-file editing is weaker

Copilot's agent mode can edit multiple files, but it doesn't match Cursor's subagent system or Kiro's spec-guided implementation. The trade-off is architectural: Copilot works through extension APIs rather than controlling the whole editor environment. It can't understand your codebase as deeply because it's a guest in someone else's house.

For quick single-file edits, this doesn't matter. For large refactoring across 10+ files, the difference is stark.

No spec workflow, no hooks

Kiro's spec-driven approach and Agent Hooks have no equivalent in Copilot. There's no way to define requirements before coding, no automated triggers on file changes, and no structured planning workflow. Copilot is reactive — it responds to what you're doing. It doesn't help you figure out what you should be doing.

Security concerns are real

Multiple reviews and studies flag that Copilot can suggest insecure code patterns. Since it learns from public repositories, it sometimes pulls in outdated or vulnerable patterns. This isn't unique to Copilot — all AI coding tools have this risk — but Copilot's shallower context awareness means it's less likely to understand your project's specific security requirements.

The Pricing Breakdown

Plan	Price	Key Features
Free	$0	2,000 completions, 50 chat/agent requests
Pro	$10/month	Unlimited completions, premium model access
Pro+	$39/month	More premium requests, coding agent
Business	$19/user/month	Organization management, policy controls
Enterprise	$39/user/month	SSO, SCIM, audit logs, IP indemnity

The free tier is genuinely useful for evaluation. Pro at $10/month is the sweet spot for individuals. But note: heavy agent usage on Pro can hit limits, pushing you toward Pro+ at $39/month — which is nearly double Cursor's flat $20.

The Three-Tool Comparison

After using all three for a week each, here's my honest ranking by category:

Category	Winner	Runner-up	Third
Inline completions	Cursor (next-edit)	Copilot	Kiro
Multi-file refactoring	Cursor (subagents)	Kiro (spec-guided)	Copilot
Planning & architecture	Kiro (specs)	Copilot (Workspace)	Cursor
IDE flexibility	Copilot (all IDEs)	—	Cursor/Kiro (VS Code only)
Codebase understanding	Cursor (deep index)	Kiro (spec context)	Copilot
Price (value)	Copilot ($10/mo)	Cursor ($20/mo)	Kiro (variable)
Ecosystem	Copilot (GitHub)	Kiro (AWS)	Cursor
Speed of small edits	Cursor	Copilot	Kiro
Code quality	Kiro (spec-driven)	Cursor	Copilot

My Verdict After 7 Days

Copilot is the Toyota Corolla of AI coding tools. It's reliable, affordable, works everywhere, and gets the job done. There's a reason 4.7 million developers pay for it.

But after experiencing Cursor's speed and Kiro's discipline, Copilot feels like it's coasting on distribution rather than innovation. The GitHub integration and IDE flexibility keep it relevant, but the core AI experience — context awareness, multi-file editing, intelligent suggestions — is falling behind.

Would I keep paying? Only if I needed JetBrains support or was on a team standardized on GitHub's ecosystem. For VS Code users, Cursor is a better tool at twice the price — and the productivity gains more than cover the difference.

Who should use it:

JetBrains users (no real alternative)
Teams already deep in the GitHub ecosystem
Developers who want the cheapest entry point
Anyone who doesn't want to switch editors

Who should look elsewhere:

VS Code users who want the best AI experience (→ Cursor)
Solo developers building features from scratch (→ Kiro)
Anyone doing heavy multi-file refactoring
Developers who want deep codebase understanding

Tips If You're Starting

Use agent mode, not just inline suggestions — the inline completions are table stakes now, the agent is where the value is
Try GPT-5.4 as your model — it's a significant upgrade over the default
Open relevant files in tabs — Copilot uses open tabs for context, so more tabs = better suggestions
Don't trust security-sensitive suggestions blindly — review anything touching auth, encryption, or user data
Consider the free tier first — 2,000 completions/month is enough to decide if it's for you

That's three weeks, three tools. My current setup: Cursor for daily coding, Kiro for new features, Copilot retired. Your mileage may vary — the best tool is the one that matches how you think, not how I think.

Originally published at https://www.aimadetools.com

Claude Code vs Cursor — Terminal Agent vs AI IDE (2026)

Joske Vermeulen — Fri, 10 Apr 2026 10:11:37 +0000

Claude Code and Cursor are the two AI coding tools developers argue about most in 2026. They represent fundamentally different philosophies: Claude Code is a terminal agent that reads your codebase and executes autonomously. Cursor is a VS Code fork with AI deeply integrated into the editing experience.

The Pragmatic Engineer's 2026 survey of nearly 1,000 developers found Claude Code is now the #1 most-used AI coding tool, overtaking both Copilot and Cursor in just eight months. But Cursor grew 35% in the same period. Both are winning — just for different developers.

The Core Difference

Claude Code = you describe what you want, the AI does it. You review the result.

Cursor = you write code with AI assistance. The AI suggests, you decide in real-time.

That's the fundamental split. Claude Code is an autonomous agent. Cursor is an augmented editor.

Feature Comparison

Feature	Claude Code	Cursor
Interface	Terminal	VS Code fork
Approach	Autonomous agent	Augmented editor
Pricing	Usage-based (~$5-20/session)	$20/mo flat (Pro)
Context window	200K (1M in beta)	Varies by model
Codebase awareness	Reads entire repo	Indexes entire project
Multi-file editing	Native (agent does it)	Composer mode
Tab completion	No	Yes (multi-line + next-edit)
Model	Claude Opus 4.6 (default)	Claude, GPT, Gemini — your pick
IDE integration	Works with any editor	Cursor only
Git integration	Can commit, push, branch	Basic
Runs commands	Yes (shell access)	Limited

Where Claude Code Wins

Autonomy

You can tell Claude Code "refactor the auth system to use JWT tokens" and walk away. It'll read the codebase, plan the changes, modify files, run tests, fix errors, and commit. Cursor's Composer is powerful, but it still expects you to be in the loop reviewing each step.

For large, well-defined tasks, Claude Code's autonomy is a massive time saver.

Context window

Claude Code runs on Opus 4.6 with a 200K context window (1M in beta). It can hold your entire codebase in context for medium-sized projects. Cursor's context is limited by whichever model you're using and how much of your project it indexes.

Works with any editor

Claude Code runs in your terminal. You can use it alongside VS Code, JetBrains, Neovim, Vim — whatever. It doesn't care about your editor. Cursor forces you into their VS Code fork.

Shell access

Claude Code can run your tests, start your dev server, check build errors, and fix them — all in the same session. It has full shell access. Cursor's terminal integration exists but the AI doesn't interact with it as naturally.

Developer love

46% of developers in the Pragmatic Engineer survey named Claude Code as the tool they love most. Cursor was at 19%. That's a significant gap in satisfaction.

Where Cursor Wins

Real-time coding flow

Cursor's Tab predictions and inline suggestions keep you in a flow state. You're writing code, and the AI is right there suggesting the next line, the next edit, the next file to change. Claude Code has no inline editing — you describe, it executes, you review. Different rhythm entirely.

If you enjoy the act of writing code (not just describing it), Cursor feels better.

Visual feedback

You see changes happening in real-time in your editor. Diffs are highlighted, you can accept or reject individual changes. With Claude Code, you see terminal output and then check the files afterward. For developers who think visually, Cursor's approach is more intuitive.

Predictable pricing

Cursor Pro is $20/month, period. Claude Code is usage-based — a heavy session can cost $5-20 depending on the model and how much context you're feeding it. If you code 8 hours a day, Claude Code can get expensive fast. Cursor's flat rate is simpler to budget.

Model flexibility

Cursor lets you switch between Claude, GPT, and Gemini models per task. Claude Code only runs Claude models. If you want GPT-5.4 for a specific task, you can't do that in Claude Code.

Pricing Reality

Claude Code

Runs on your Anthropic API key or Claude Max subscription
Claude Max: $100/mo (5x usage), $200/mo (20x usage)
API: ~$5-15 per heavy coding session (varies wildly)
No free tier for coding use

Cursor

Free: 2,000 completions, 50 premium requests
Pro ($20/mo): Unlimited completions, 500 premium requests
Business ($40/mo): Team features

For light-to-moderate use, Cursor is cheaper. For heavy autonomous work, Claude Code can cost more but potentially saves more time.

Who Should Use What

Choose Claude Code if:

You're comfortable in the terminal
You want maximum autonomy (describe → AI builds)
You work on large refactoring tasks
You already pay for Claude Max
You use a non-VS Code editor

Choose Cursor if:

You love the VS Code editing experience
You want real-time AI suggestions while you type
You prefer predictable monthly pricing
You want to choose between multiple AI models
You enjoy hands-on coding with AI assistance

The power move: Use both. Claude Code for big autonomous tasks ("refactor this entire module"), Cursor for daily editing with inline suggestions. Many developers in the Pragmatic Engineer survey reported using 2-4 AI tools simultaneously.

Claude Code is next on my I Used It for a Week review list. Stay tuned.

Related: Best AI Coding Tools in 2026: The Definitive Ranking

Originally published at https://www.aimadetools.com