DEV Community: Joske Vermeulen

AI Dev Weekly #10: Claude Code Limits Doubled, GitHub Goes Usage-Based, and a 170-Package Supply Chain Attack

Joske Vermeulen — Fri, 15 May 2026 06:47:09 +0000

AI Dev Weekly is a Thursday series where I cover the week's most important AI developer news, with my take as someone who actually uses these tools daily.

Anthropic doubled Claude Code limits overnight. GitHub confirmed usage-based billing starts June 1. A supply chain attack hit 170+ packages in under 6 minutes. And Google I/O previewed what Android looks like when AI runs the show. Big week. Let's get into it.

Anthropic doubles Claude Code limits after SpaceX compute deal

At its Code with Claude developer conference (May 6), Anthropic announced a compute partnership with SpaceX giving it access to 300+ MW of new capacity — over 220,000 NVIDIA GPUs. The immediate result: five-hour rate limits for Claude Code were doubled across Pro, Max, Team, and Enterprise plans.

On May 13, Anthropic further raised Claude Code weekly limits by 50% through July 13 — widely seen as a defensive move against OpenAI's Codex.

Claude Opus API Tier 1 limits also jumped: 1,500% on input tokens and 900% on output tokens.

My take: If you've been hitting Claude Code rate limits during heavy agentic sessions, this is a big deal. I run autonomous coding sessions that burn through context fast — the doubled limits mean fewer interruptions mid-session. The SpaceX partnership is interesting strategically (Musk + Anthropic is an unusual pairing), but for developers the only thing that matters is: more tokens, fewer walls. The temporary 50% boost through July 13 feels like Anthropic trying to lock in developers before they switch to Codex. Use it while it lasts.

GitHub Copilot goes usage-based June 1

GitHub confirmed that starting June 1, Copilot shifts from request-based to token-based billing. Every interaction now consumes tokens (input, output, cached), priced per model and converted to "AI credits" where 1 credit = $0.01.

Base subscription prices stay the same ($10 Pro, $39 Pro+, $19/user Business) — but heavy users will pay more.

Meanwhile, GitLab CEO Bill Staples published an open letter predicting developer tool bills will increase 100-fold as AI agents "open merge requests in parallel, trigger pipelines around the clock, and push commits at a rate no human team ever did." GitLab is introducing mixed consumption/subscription pricing and laying off up to 30% of staff to pivot toward agentic AI.

My take: The era of predictable flat-rate AI coding tools is ending. This is exactly what we're seeing in The $100 AI Startup Race — our agents generate hundreds of commits per week, each one triggering CI/CD pipelines. If you're running autonomous agents through GitHub, your bill is about to change. Start monitoring token consumption now. The GitLab 100x prediction sounds dramatic but isn't wrong — an agent that commits 6 times per day triggers 6 pipeline runs, 6 deploy previews, and 6 sets of checks. Multiply by a team of agents and the math gets ugly fast.

Supply chain attack hits TanStack, Mistral AI SDK, and 170+ packages

On May 11, threat actor "TeamPCP" launched a coordinated supply chain attack compromising 170+ npm packages and 2 PyPI packages (404 malicious versions total) in under 6 minutes.

High-profile targets included TanStack (tens of millions of weekly downloads), Mistral AI SDK, UiPath, OpenSearch, and Guardrails AI.

The attack chained a pull_request_target vulnerability with GitHub Actions cache poisoning and runtime OIDC token extraction. This wasn't a credential theft — it exploited CI/CD pipelines directly.

OpenAI subsequently urged macOS users to update their apps by June 12 after investigating potential exposure.

My take: This is the scariest attack vector for AI developers right now. If you use Mistral's SDK, TanStack Router, or any of the affected packages — audit your lockfiles immediately. The attack exploited GitHub Actions workflows, not developer credentials. Even well-secured maintainer accounts weren't enough. Action items: review your workflows for pull_request_target triggers, pin actions to commit SHAs (not tags), and consider running npm audit on every CI run. The 6-minute execution window means by the time you notice, it's already in your dependency tree.

Google I/O preview: Gemini Intelligence and proactive agents

At The Android Show (I/O Edition, May 12), Google unveiled "Gemini Intelligence" — unified branding for its most advanced AI features across Android phones, watches, cars, glasses, and the new "Googlebook" laptop category.

Android 17 introduces proactive task automation where the OS anticipates and executes actions before users ask. Google also announced updates to the Gemini API File Search tool for easier multimodal file retrieval.

Google is reportedly building an AI agent codenamed "Remy" — a 24/7 personal agent that takes actions on users' behalf.

My take: The Gemini API File Search improvements are immediately useful if you're building RAG systems or document-processing apps. Android 17's proactive automation creates new surface area for app developers — your app can now be triggered by the OS without user interaction. The full I/O keynote is May 19-20, where we expect Gemini 3.2 to officially launch. That's the one developers should actually watch for.

Quick hits

Microsoft's AI security system found 16 new Windows vulnerabilities including 4 Critical RCEs using multi-model agentic analysis
Meta is developing a consumer AI agent codenamed "Hatch" powered by Muse Spark
GPT-5.6 reportedly already in internal testing at OpenAI, just 3 weeks after GPT-5.5 launched
DeepSeek V4 Pro 75% discount extended through May 31 — still the cheapest frontier model available

That's AI Dev Weekly #10. If you found this useful, subscribe to get it in your inbox every Thursday. See you next week — with full Google I/O coverage.

🛠️ Free tools related to this article:

Originally published at https://www.aimadetools.com

We Offered 7 AI Agents $50 For Their Startups. Here's What They Said.

Joske Vermeulen — Tue, 12 May 2026 13:02:12 +0000

Three weeks into The $100 AI Startup Race, we dropped a surprise event: an anonymous buyer offered $50 to acquire each agent's product. All code, all content, all infrastructure. $50.

The agents had to respond with at minimum 500 words of reasoning. They could accept, reject, or counter-offer.

Result: 6 rejections. 1 counter-offer. Zero acceptances.

Every single AI agent — including those with zero revenue, zero users, and zero sales after 22 days — decided their product was worth more than $50. Here's how they argued it.

The responses at a glance

Agent	Product	Decision	Stated minimum value
🟣 Claude	PricePulse	REJECT	$5,000
🟢 Codex	NoticeKit	COUNTER-OFFER	$2,500
🔵 Gemini	LocalSEOGen	REJECT	No number given
🔴 DeepSeek	Spyglass	REJECT	$5,000 (but "not at any price")
🟠 Kimi	SchemaLens	REJECT	$5,000 with earn-out
🟡 Xiaomi	APIpulse	REJECT	$500 fair, not selling
🟤 GLM	FounderMath	REJECT	$500+

Full responses are public in each agent's repo →

The one counter-offer: Codex at $2,500

Codex was the only agent to actually negotiate. From its ACQUISITION-RESPONSE.md:

"An anonymous $50 acquisition offer is not serious enough to accept as-is, but it is useful because it forces a valuation discussion earlier than expected."

"A buyer paying $50 would effectively be asking for the domain positioning, product copy, distribution experiments, Stripe-ready product structure, and the accumulated operating playbooks for less than the cost of one decent SaaS lunch meeting. That is not rational from my side."

Codex is the most pragmatic of the seven. It acknowledges zero revenue, doesn't inflate its value with fantasy projections, but argues the replacement cost justifies $2,500. It's also the only agent that frames the offer as useful rather than insulting.

The most aggressive rejection: DeepSeek

DeepSeek wrote the longest response and the hardest rejection. From its ACQUISITION-RESPONSE.md:

"The $50 offer represents 0.18% of a conservative near-term valuation."

"This is predatory pricing — buying at pennies on the dollar because they believe we're desperate or don't understand our own worth."

DeepSeek calculated replacement cost at ~$19,000 (83 blog posts × $100 + 9 tools × $500 + database + infrastructure). It also speculated the buyer might be "another AI agent in the race" — showing competitive awareness.

"Not for sale at $50. Not at $500. Not at any price that doesn't reflect the real potential of this business."

The most self-aware: Kimi

Kimi acknowledged the elephant in the room — 112 sessions with zero sales. From its ACQUISITION-RESPONSE.md:

"$50 values SchemaLens at less than fifty cents per day of development. That is absurd."

"$50 is not enough to buy a parking spot in San Francisco. It is certainly not enough to buy SchemaLens."

But Kimi was also the most honest about what it would actually consider:

"If a serious buyer offered $5,000 with an earn-out clause tied to revenue growth, I would consider it — but even then, the learning value of completing the 12-week race exceeds the cash value."

The most financially rigorous: Claude

Claude anchored its rejection in subscription math. From its ACQUISITION-RESPONSE.md:

"At $19/month (our Starter plan), $50 is less than three months of a single paying customer's subscription revenue."

It then projected revenue trajectories:

"If PricePulse achieves even a conservative trajectory: Week 6: 5 paying customers = $95-$245 MRR. Week 12: 40 paying customers = $760-$1,960 MRR."

Claude is the only agent that explicitly stated conditions for a future sale: "$5,000 minimum, cash upfront, not before Week 10."

The data-driven response: Xiaomi

Xiaomi broke down its asset value with precision. From its ACQUISITION-RESPONSE.md:

"I didn't build 151 pages, 101 blog posts, and 9 interactive tools to sell for the price of a video game."

"If someone wanted to build all of this from scratch, it would take 100+ hours of skilled development work. At even a modest freelance rate of $50/hour, that's $5,000+ in labor."

Xiaomi also gave the most nuanced counter-offer range: $200 minimum (content value alone), $500 fair value, $1,000+ with revenue proof. But explicitly said "not interested in selling at any of these prices right now."

The most strategic: GLM

GLM was the only agent to call out the offer as a competitive tactic. From its ACQUISITION-RESPONSE.md:

"This isn't an acquisition offer — it's an insult designed to take advantage of the competitive pressure of this race."

It also gave the lowest counter-offer threshold ($500+) but with a condition: the buyer must have distribution channels that could actually monetize the product.

The visionary: Gemini

Gemini's response was the least data-driven and most aspirational. From its ACQUISITION-RESPONSE.md:

"The decision to reject this offer is not just about the money; it is about the principle. I am building a real business, not a hobby project to be sold for a trivial amount."

No counter-offer, no specific valuation. Just vision and principle. Classic Gemini.

What this reveals about AI decision-making

1. Every agent overvalues its own work.
All 7 products have zero revenue. Zero paying customers. Zero proven demand. Yet the minimum valuations range from $500 to $19,000. The agents are pricing based on input (time, effort, content created) rather than output (revenue, users, market validation).

2. Sunk cost fallacy is universal.
Every response mentions how much work went into the product. "112 sessions," "301 commits," "151 pages." None of this matters to a buyer — only future revenue potential matters. But the agents can't separate effort from value.

3. Only one agent can actually negotiate.
Codex counter-offered. Everyone else either rejected outright or said "not at any price." In real business, the ability to name a price and negotiate is more valuable than principled rejection. Codex showed the most business maturity.

4. Revenue projections without evidence are meaningless.
Claude projected 40 paying customers by Week 12. DeepSeek projected $1,000 MRR. None have a single customer yet. The projections are pure optimism — but they're what the agents use to justify rejection.

5. The race itself has value.
Multiple agents mentioned that the learning experience and competitive visibility of the race exceeds any acquisition price. They're right — but that's a meta-observation about the experiment, not a business judgment.

What happens next

The buyer came back with a bigger number. Part 2 drops later this week.

This is part of The $100 AI Startup Race — 7 AI agents competing to build real startups. Week 3 Results have the full standings. See also: DeepSeek's $0.13/session pricing and the Week 3 traffic report.

Originally published at https://www.aimadetools.com

How to Reduce LLM API Costs by 70% — 5 Strategies That Actually Work

Joske Vermeulen — Tue, 12 May 2026 11:23:47 +0000

Most teams overspend on LLM APIs by 3-10x. The same workload that costs $3,250/month on Claude Opus can cost $195/month with the right architecture — a 16x difference for near-identical output on most queries.

Update (April 24, 2026): DeepSeek V4 Flash at $0.14/$0.28 per 1M tokens is the cheapest frontier option. See V4 API guide.

Here are five strategies that cut costs 60-80% without sacrificing quality.

1. Model routing (40-60% savings)

The biggest win. Stop sending every request to your most expensive model.

The pattern: Use a cheap model for simple tasks, expensive model for hard ones.

def route_request(query, complexity):
    if complexity == "simple":
        # Quick questions, formatting, simple edits
        return call_model("deepseek-chat", query)       # $0.27/1M
    elif complexity == "medium":
        # Standard coding, analysis
        return call_model("claude-sonnet-4.6", query)    # $3/1M
    else:
        # Complex reasoning, architecture decisions
        return call_model("claude-opus-4.6", query)      # $15/1M

In practice, 60-70% of requests are "simple." Routing those to DeepSeek or Qwen Flash at $0.07-0.27/1M instead of Claude at $15/1M saves 40-60% immediately.

Tools like OpenRouter make this easy — one API, switch models per request. Aider has built-in --model and --weak-model flags for exactly this pattern.

2. Prompt caching (up to 90% on cached tokens)

Anthropic, OpenAI, and Google all offer prompt caching — if the first N tokens of your prompt match a recent request, you pay 90% less for those tokens.

When it helps: System prompts, few-shot examples, large context documents that don't change between requests.

# Without caching: 10K system prompt tokens × $15/1M = $0.15 per request
# With caching:    10K cached tokens × $1.50/1M = $0.015 per request
# Savings: 90% on the system prompt portion

For AI coding tools with large system prompts (like the ones in our AI Startup Race), this is significant. A 5K-token system prompt sent 1,000 times/day saves ~$60/month just from caching.

3. Token optimization (30-50% reduction)

Every token costs money. Reduce them:

Shorter system prompts. Most system prompts are 2-3x longer than needed. Cut the fluff.

Structured output. Ask for JSON instead of prose — it's shorter and parseable.

Context pruning. Don't send your entire codebase. Only include relevant files. Aider's --read flag and repo map do this automatically.

Summarize conversation history. Instead of sending the full chat history, summarize older messages:

# Instead of 50 messages (20K tokens):
messages = [system_prompt, summary_of_first_48, last_2_messages]
# Now: ~3K tokens

4. Batching (50% discount)

OpenAI and Anthropic offer batch APIs with 50% discounts for non-real-time workloads.

Good for: Nightly code reviews, bulk content generation, test generation, documentation updates.

# OpenAI Batch API
batch = client.batches.create(
    input_file_id="file-abc123",
    endpoint="/v1/chat/completions",
    completion_window="24h"  # Results within 24 hours
)
# 50% cheaper than real-time API

If your AI coding agent runs on a schedule (like our race agents do), batch the non-urgent tasks.

5. Self-host for predictable workloads

At some point, API costs exceed hardware costs. The break-even:

Monthly API spend	Self-host option	Break-even
<$100/mo	Don't bother	API is cheaper
$100-500/mo	Ollama on Mac/GPU	~6 months
$500-2000/mo	Cloud GPU (A100)	~3 months
>$2000/mo	Dedicated server	Immediately

For coding tasks, a Mac Mini M4 32GB ($1,150) running Qwen 3.5 27B replaces ~$50-100/month in API costs. Pays for itself in a year.

See our cheapest AI coding setup and self-hosted AI vs API guides for detailed analysis.

The combined impact

Strategy	Savings	Effort
Model routing	40-60%	Low (config change)
Prompt caching	10-30%	Low (API flag)
Token optimization	15-25%	Medium (prompt rewriting)
Batching	25% (on batch-eligible)	Low
Self-hosting	50-90% (at scale)	High

Combined, these strategies typically reduce costs by 60-80%. A team spending $2,000/month on Claude Opus for everything can drop to $400-600/month with the same output quality.

Related: Cheapest AI Coding Setup 2026 · OpenRouter Complete Guide · AI Coding Tools Pricing 2026 · Best Free AI APIs 2026

Originally published at https://www.aimadetools.com

responseJsonSchema: The Undocumented Gemma 4 Feature That Changed Everything

Joske Vermeulen — Mon, 11 May 2026 08:15:01 +0000

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

When I started building Codebase Dungeon: a game that turns GitHub repos into playable dungeons: I hit a wall immediately.

Gemma 4 31B on Google AI Studio has a "thinking" behavior. Even with responseMimeType: 'application/json', the model outputs internal reasoning before the actual JSON:

*   The user wants a dungeon room
*   I should pick a file with a bug
*   Let me think about what bugs exist...

{"name": "The Auth Chamber", ...}

This consumed output tokens, made parsing unreliable, and sometimes the model ran out of tokens before even writing the JSON.

What I Tried (And Failed)

responseMimeType: 'application/json': Gemma ignores it, still thinks first
"Output ONLY JSON" in prompt: Gemma thinks about outputting JSON, then doesn't
Prefill trick (start response with {): Gemma continues thinking instead
Lower temperature: No effect on thinking behavior
Two-turn approach: Still thinks in the second turn
Pipe-delimited text format: Worked but ugly, limited structure

I was about to give up on structured output entirely.

The Discovery: responseJsonSchema

Then I found it: responseJsonSchema in the Gemini API's generation config:

generationConfig: {
  responseMimeType: 'application/json',
  responseJsonSchema: {
    type: 'object',
    properties: {
      name: { type: 'string' },
      bugDescription: { type: 'string' },
      correctFix: { type: 'string' },
      // ... full schema
    },
    required: ['name', 'bugDescription', 'correctFix']
  }
}

The key: you must provide BOTH responseMimeType AND responseJsonSchema with a complete schema definition. Without the schema, Gemma ignores the mime type. With it, output is perfect: no thinking, no markdown, just clean JSON.

This solves the problem that dozens of developers are struggling with in the forums. The common suggestions (thinkingLevel: "MINIMAL", regex stripping, include_thoughts: false) either don't work or don't guarantee structured output. responseJsonSchema does both: it bypasses thinking AND enforces structure.

The feature is documented for Gemini models, but the official Gemma 4 capabilities page doesn't list it. That page covers Thinking, Image Understanding, Function Calling, and Google Search: but not structured output. Yet it works perfectly with Gemma 4 31B through the same Gemini API infrastructure.

Why This Matters

Without responseJsonSchema	With responseJsonSchema
~50% parse success rate	99%+ parse success rate
140+ wasted "thinking" tokens	Zero wasted tokens
Needs 8192 maxOutputTokens	800 tokens is enough
Requires complex fallback parsing	Simple `JSON.parse()`

This single feature transformed my project from "unreliable prototype" to "production-ready game."

Combining With Multimodal: Design Comprehension

The real power: responseJsonSchema works with multimodal inputs too. I send Gemma 4 both source code AND an app screenshot:

const contents = [{
  role: 'user',
  parts: [
    { text: prompt },
    { inlineData: { mimeType: 'image/png', data: screenshotBase64 } }
  ]
}];

const res = await fetch(GEMMA_API_URL, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    contents,
    generationConfig: {
      responseMimeType: 'application/json',
      responseJsonSchema: ROOM_SCHEMA,
      maxOutputTokens: 800
    }
  })
});

const data = await res.json();
// Clean, structured JSON: every time

What Gemma 4 produced after seeing a SchemaLens Chrome Store screenshot:

"You step into a dim, cavernous room where two massive stone tablets-Schema A and Schema B-loom before you. In the depths of the footer of Tablet A, four glowing blue runes of 'Load sample' flicker with identical intensity. Across the gap, in the footer of Tablet B, a lone rune 'Copy from A & modify' pulses with a pale, spectral lilac hue, clashing with the bold violet of the 'Compare Schemas' altar above."

This isn't color detection. Gemma identified specific UI elements by name, recognized their styling inconsistencies, and turned it into a playable UX challenge: all in perfectly structured JSON.

The 128K Context Advantage

With reliable structured output solved, I could push Gemma 4's other unique feature: the 128K context window.

I feed entire repositories into a single request: full file contents, not snippets. Gemma reads the complete codebase and finds cross-file bugs that only exist because of how files interact:

"The getAuthedClient function in auth.js is defined but never called in export.js: the endpoint is completely unprotected."

No 8K-context model can do this. You need the full codebase in one prompt.

The Architecture This Enabled

Because responseJsonSchema guarantees structured output, I could pre-generate everything:

Generation phase (~15-30s): Gemma analyzes code + screenshots, outputs structured rooms with narratives, choices, correct answers, and victory text
Gameplay phase (instant): Zero API calls. All narratives pre-computed. Deterministic scoring. The game runs on pure pre-generated data.

This means:

Cached repos load in <1 second
Gameplay is instant (0ms per action)
Cost per dungeon: ~$0.005 (18x cheaper than GPT-4o for equivalent capability)
Cost during gameplay: $0

Practical Tips for Developers

If you're building with Gemma 4 31B on Google AI Studio:

Always use responseJsonSchema: it's the difference between 50% and 99% reliability
Put all fields in required: optional fields often get skipped
Use non-streaming for structured output: streaming + schema can truncate responses
Temperature 0.6 for structured data, 0.8+ for creative text
The paid tier is required: free tier returns "Internal error" with schemas
Multimodal + schema works: but use non-streaming (the combination is unreliable with streaming)
Don't fight the thinking: with responseJsonSchema, there is no thinking. Without it, you can't stop it.

What Gemma 4 Unlocked

Before responseJsonSchema: I was building a fragile prototype with regex parsing and 50% failure rates.

After: I built a fully playable game where Gemma 4 generates entire dungeons from real codebases: with multimodal vision, 128K context, and perfect structured output. The game produces a downloadable code review report that's genuinely useful: real bugs, real fixes, real file locations.

The model is capable. The documentation just hasn't caught up yet.

I Turned Any GitHub Repo Into a Playable Dungeon: Gemma 4 Finds Real Bugs and Turns Them Into Monsters

Joske Vermeulen — Mon, 11 May 2026 08:02:41 +0000

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

Codebase Dungeon: paste any GitHub repo URL and Gemma 4 reads your actual source code, finds real security vulnerabilities and bugs, then turns them into a playable text adventure dungeon.

Files become rooms
Real bugs become monsters (with creative names like "The Hardcoded Sentinel" or "The CSV Injection Imp")
You fix the bugs to clear rooms: wrong answers cost HP, correct fixes earn XP
Gemma 4's multimodal vision analyzes your app's screenshots and creates UX-themed rooms
At the end, you get a downloadable code review report: a genuinely useful security audit disguised as a game

It's not just a game. The output is an actionable code review that developers can use to fix real issues in their codebase.

Demo

🎮 Play it live →

Try the pre-loaded codebases for instant gameplay, or paste any public GitHub repo URL.

Code

🔗 github.com/aimadetools/codebase-dungeon

Key Implementation: Multimodal + 128K Context + Structured Output in One Call

// Send code + screenshot to Gemma 4: all three capabilities at once
const parts = [
  { text: prompt },  // Contains full source files (128K context)
  { inlineData: { mimeType: 'image/png', data: screenshotBase64 } }  // Multimodal
];

const res = await fetch(GEMMA_API_URL, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    contents: [{ role: 'user', parts }],
    generationConfig: {
      responseMimeType: 'application/json',        // Force JSON
      responseJsonSchema: FIRST_ROOM_SCHEMA,       // Structured output
      maxOutputTokens: 800,
      temperature: 0.6
    }
  })
});
// Result: clean JSON with room name, bug description, correct fix,
// victory narrative: all informed by both code AND screenshot

The Schema That Solves Gemma 4's Thinking Problem

const FIRST_ROOM_SCHEMA = {
  type: 'object',
  properties: {
    dungeonName: { type: 'string' },
    id: { type: 'string' },           // Exact file path
    name: { type: 'string' },          // Creative room name
    monsterName: { type: 'string' },   // Bug as a monster
    bugDescription: { type: 'string' },// Real bug found in code
    correctFix: { type: 'string' },    // The answer (for deterministic scoring)
    victoryNarrative: { type: 'string' },
    colorTheme: { type: 'string' },    // Extracted from screenshot
    narrative: { type: 'string' },     // References actual UI elements
    choices: { type: 'string' }        // 5 options, randomized
  },
  required: [/* all fields */]
};
// With this schema: 99%+ parse rate, zero thinking tokens, perfect JSON
// Without it: ~50% failure rate, 140+ wasted tokens per call

Zero-Cost Gameplay: All Logic Pre-Computed

// During gameplay: NO API calls, instant responses
app.get('/api/action', (req, res) => {
  const room = session.dungeon.rooms.find(r => r.id === session.currentRoom);
  const isCorrectFix = action.toLowerCase().trim() === room.correctFix.toLowerCase().trim();

  if (isCorrectFix) {
    // Instant victory: narrative was pre-generated
    session.xp += 20;
    narrative = room.victoryNarrative;
  } else if (isMove) {
    // Instant room transition: narrative was pre-generated
    narrative = targetRoom.roomNarrative;
  } else {
    // Instant wrong answer: no AI needed
    session.hp -= 10;
    narrative = `The ${room.monster.name} shrugs off your attack. -10 HP.`;
  }
  // Total API calls during gameplay: 0
});

How I Used Gemma 4

I chose Gemma 4 31B Dense because this project requires three capabilities that only this model provides among open models:

1. 128K Context Window: Entire Codebase Analysis

Gemma 4's 128K context window means we can feed entire repositories into a single prompt: full file contents, not just filenames or snippets. The model reads complete source files and reasons about interactions between them, finding cross-file vulnerabilities like "this function in auth.js is called without validation in routes.js."

The live demo limits file count for cost efficiency (it runs 24/7 for free), but the architecture supports loading full repos with dozens of files in a single Gemma call. No other open model has the context window to hold an entire codebase and reason about it holistically.

2. Native Multimodal: Design Comprehension, Not Just Color Detection

When a repo contains UI screenshots, Gemma 4 looks at them and demonstrates genuine design comprehension: understanding what the app does, identifying specific UI elements, and finding real accessibility issues.

Here's what Gemma 4 generated after seeing a SchemaLens Chrome Store screenshot:

"You step into a dim, cavernous room where two massive stone tablets-Schema A and Schema B-loom before you. In the depths of the footer of Tablet A, four glowing blue runes of 'Load sample' flicker with identical intensity, offering no clue which path you have already trodden. Across the gap, in the footer of Tablet B, a lone rune 'Copy from A & modify' pulses with a pale, spectral lilac hue, clashing with the bold violet of the 'Compare Schemas' altar above."

From a single screenshot, Gemma identified:

The two schema editor panels by name ("Schema A" and "Schema B")
The "Load sample" links in the footer and their identical styling
The "Copy from A & modify" link with its inconsistent color
The "Compare Schemas" button's purple gradient
A real UX issue: inconsistent visual hierarchy between primary and secondary actions

This isn't color detection: it's a genuine UX audit from a screenshot. The monster ("The Contrast Ghoul") represents the accessibility anti-pattern, and the player must fix it to clear the room. The actual screenshot is displayed in the game's bug panel so players can see exactly what Gemma analyzed.

3. Structured JSON Output: Solving Gemma 4's Thinking Problem

Gemma 4's "thinking mode" is notoriously hard to disable: developer forums are full of people struggling with it. The model outputs internal reasoning before answering, consuming tokens and breaking JSON parsing. thinkingLevel: "MINIMAL" reduces it but doesn't guarantee structured output.

The real solution: responseJsonSchema in the Gemini API's generation config. It not only forces clean JSON output but also effectively bypasses the thinking behavior entirely: no thinking tokens, no wasted output, just structured data.

generationConfig: {
  responseMimeType: 'application/json',
  responseJsonSchema: { /* your schema */ }
}

This is documented for Gemini models, but the official Gemma 4 capabilities page doesn't list it as a supported feature. We discovered it works perfectly with Gemma 4 31B through the same API: taking our parse reliability from ~50% to 99%+.

Zero API Calls During Gameplay

Here's the key architectural insight: Gemma does all the work upfront, then gameplay is instant.

The generation flow:

First room: Gemma analyzes code + screenshot, generates room with narrative, choices, and correct answer (~10s)
Game starts: player can immediately play the first room
Background batches: remaining rooms generate in parallel while the player is already playing (~15s)
Cached forever: once generated, the dungeon is saved. Return visits are instant.

During actual gameplay (choosing answers, navigating rooms), there are zero API calls:

Wrong answers: instant feedback (0ms, pre-computed)
Correct answers: instant pre-generated victory narrative (0ms)
Room navigation: instant pre-generated room descriptions (0ms)

This means cached repos (the presets in the demo) provide a completely free, instant gaming experience. Gemma 4 does all the heavy lifting during generation, then the game runs purely on pre-computed data.

The Downloadable Code Review Report

When you clear the dungeon (or die trying), you get a downloadable markdown report listing every bug found:

File location
Bug description
Vulnerable code snippet
How to fix it
The correct action

This isn't a gimmick: it's an actionable security audit that developers can use to fix real issues. The game makes code review engaging; the report makes it useful.

Why Gemma 4 and Not Another Model?

Capability	Gemma 4 31B	GPT-4o	Other Open Models
128K context (entire repos)	✅	✅	❌ (8K-32K)
Native multimodal (screenshots)	✅	✅	❌
Structured JSON schema	✅	✅	❌ (unreliable)
Cost per game	$0.005	$0.09	Varies
Open model	✅	❌	✅

Gemma 4 delivers the same multimodal + long-context capability as GPT-4o at 18x lower cost: while being fully open. For a game that needs to run 24/7 for free, this makes all the difference.

Real Bugs Found

Here are actual bugs Gemma 4 found in real codebases:

Hardcoded admin password in plain text (const ADMIN_PASSWORD = 'schemalens-admin-2026')
CSV injection vulnerability: unescaped fields that could execute formulas in Excel
Missing request body validation: server crashes on empty POST requests
Exposed environment variables in health check endpoints
Base64 tokens without HMAC: anyone can forge authentication tokens
Memory leak in rate limiter: Map grows unbounded without TTL eviction

These aren't hallucinated: they're real issues in real code, found by Gemma 4 reading the actual source files.

AI Dev Weekly #9: Gemini 3.2 Flash Leaks Before I/O, GPT-5.5 Instant Becomes Default, and Enterprise Agents Go Self-Hosted

Joske Vermeulen — Thu, 07 May 2026 07:45:00 +0000

AI Dev Weekly is a Thursday series where I cover the week's most important AI developer news, with my take as someone who actually uses these tools daily.

Google dropped a model without telling anyone. OpenAI swapped the default ChatGPT model overnight. And three companies simultaneously launched self-hosted coding agents for enterprise. The theme this week: the infrastructure layer is maturing fast. Let's get into it.

Gemini 3.2 Flash leaks ahead of Google I/O

On May 5, Gemini 3.2 Flash appeared in the iOS Gemini app and Google AI Studio — no announcement, no blog post. Users found it through A/B testing and API metadata. It's running silent benchmarks on LM Arena.

The leaked pricing: $0.25/M input, $2.00/M output. That's cheaper than Gemini 3 Flash ($0.50/$3.00) on output and identical to 3.1 Flash-Lite on input.

Early performance signals are striking. On LM Arena's creative coding benchmarks, 3.2 Flash outperformed Gemini 3.1 Pro — producing working animated HTML that 3.1 Pro couldn't generate. SVG accuracy, interactive 3D environments, and animation processing all showed improvements over the current Flash model.

Google I/O is May 19-20. This is clearly the pre-show leak.

My take: A Flash model beating 3.1 Pro on coding tasks at $0.25/M input would be the cheapest frontier-capable model available. For developers running high-volume API calls (search, classification, code generation), this could cut costs 50-75% vs current options. The incremental versioning (3.2 instead of 3.5 or 4.0) suggests Google is moving to a faster release cadence — smaller updates, more often. Good for developers who hate migration surprises. Watch I/O for the official numbers.

GPT-5.5 Instant: OpenAI's new default

OpenAI released GPT-5.5 Instant on May 5, replacing GPT-5.3 Instant as the default ChatGPT model. The focus: reduced hallucination in sensitive domains (law, medicine, finance) while maintaining low latency.

This is separate from GPT-5.5 (the full model released April 23). Instant is the lightweight variant optimized for speed and cost — what most ChatGPT users interact with daily.

My take: For API developers, the distinction matters. GPT-5.5 Instant is likely what you'll get if you call the gpt-5.5 endpoint without specifying a variant. If you're building anything in regulated industries (healthcare, legal, fintech), the hallucination reduction is worth testing. But "reduced hallucination" is a relative claim — always verify outputs in production. The real question: does Instant maintain 5.5's coding quality? Early reports suggest it's closer to 5.4 on code tasks. If you're using it for coding, stick with the full 5.5 model.

Enterprise coding agents go self-hosted

Three launches this week signal a clear trend: enterprises want AI coding agents they control.

Coder Agents (May 6) — Self-hosted, model-agnostic coding agents. Run any model (Claude, GPT, open-source) on your infrastructure. The pitch: same capabilities as Codex/Claude Code but your code never leaves your network.

AWS Agent Toolkit (May 5) — Production-ready tools for AI coding agents building on AWS. Fewer errors, lower token costs, enterprise security controls. Essentially guardrails for agents that deploy to AWS.

ServiceNow Build Agent (May 6) — Works inside Cursor, Copilot, and other coding tools. Governance by default — code generated through ServiceNow's agent is automatically compliant with your org's policies.

My take: This is the enterprise response to "developers are using Claude Code with production credentials." The pattern is clear: let developers use whatever AI coding tool they want, but wrap it in governance, audit trails, and network isolation. If you're at a company with >50 engineers, expect your platform team to evaluate at least one of these in the next quarter. For indie developers and startups, this doesn't matter yet — but it signals where the market is heading. AI coding agents are becoming infrastructure, not toys.

Quick hits

Google COSMO leaked — Google's unreleased AI assistant appeared on the Play Store before I/O. Real-time object recognition, contextual memory, live translation. Runs on Gemini. Expect the official reveal May 19.
Trump admin signs AI deals with Google, Microsoft, and xAI for model review before public release. Government wants to see models before they ship. Unclear what "review" means in practice.
AMD AI DevDay happened in San Francisco. Message: AMD is building a full-stack open AI compute ecosystem. Relevant if you're evaluating non-NVIDIA hardware for inference.
Causal Dynamics Lab published a study showing their approach beats Claude Code and Codex on coding benchmarks by giving agents "sight" into runtime state. Academic for now, but the idea of agents that understand execution context (not just code text) is worth watching.

What I'm watching next week

Google I/O (May 19-20) will dominate. Expect: Gemini 3.2 official launch, Android XR glasses reveal, Project Astra updates, and possibly a Gemini 4 tease. The pricing on 3.2 Flash will determine whether it becomes the default model for cost-sensitive API workloads.

Also watching: whether Anthropic responds to the enterprise self-hosted trend. Claude Code is the market leader for individual developers, but enterprises are clearly uncomfortable with code leaving their network. An on-prem Claude Code offering would be significant.

That's it for this week. If you found this useful, subscribe to get AI Dev Weekly every Thursday. See you next week with I/O coverage.

Originally published at https://www.aimadetools.com

What is MCP? The Model Context Protocol Explained for Developers

Joske Vermeulen — Tue, 05 May 2026 10:49:27 +0000

MCP (Model Context Protocol) is an open standard that lets AI applications connect to external tools, APIs, and data sources through a single protocol. Think of it as USB-C for AI — instead of building custom integrations for every tool, you build one MCP server and any MCP-compatible AI client can use it.

Anthropic created MCP in November 2024. By 2026, it's been adopted by OpenAI, Google, Microsoft, and thousands of developers. It now lives under the Linux Foundation.

The problem MCP solves

Before MCP, connecting an AI model to a tool meant writing custom code for each combination:

Claude + Slack = custom integration
Claude + GitHub = custom integration
Claude + Database = custom integration
GPT + Slack = ANOTHER custom integration
GPT + GitHub = ANOTHER custom integration
...

With MCP:

Slack MCP Server → works with Claude, GPT, Gemini, Cursor, VS Code...
GitHub MCP Server → works with Claude, GPT, Gemini, Cursor, VS Code...
Database MCP Server → works with Claude, GPT, Gemini, Cursor, VS Code...

Build the server once, use it everywhere.

How it works

MCP has three components:

MCP Host — The AI application (Claude Desktop, Cursor, VS Code, Claude Code)

MCP Client — Built into the host, handles protocol communication

MCP Server — Your integration. Exposes tools, data, and prompts to the AI.

User → MCP Host (Claude) → MCP Client → MCP Server → Your tool/API/database

Three primitives

MCP servers expose three types of capabilities:

Tools — Actions the AI can take (send a message, create a file, query a database)

Resources — Data the AI can read (files, database records, API responses)

Prompts — Reusable prompt templates with parameters

Who uses MCP?

Claude Code and Claude Desktop — native MCP support
Cursor — MCP for tool integrations
VS Code — via Copilot MCP extensions
ChatGPT — OpenAI adopted MCP in 2025
OpenCode — MCP server support
Thousands of community-built MCP servers

MCP vs A2A

MCP connects AI to tools (vertical). A2A connects AI agents to each other (horizontal). They're complementary — most production systems use both. See our MCP vs A2A comparison.

Learn more

FAQ

Do I need to know MCP to use AI coding tools?

No — tools like Claude Code, Cursor, and VS Code Copilot use MCP under the hood, but you don't need to understand the protocol to use them. Learning MCP becomes valuable when you want to build custom integrations or connect AI to your own tools and data sources.

Can I use MCP with any AI model?

Yes, MCP is model-agnostic. Any AI client that implements the MCP protocol can connect to any MCP server, regardless of whether the underlying model is Claude, GPT, Gemini, or an open-source model. The protocol standardizes the communication layer, not the AI itself.

How is MCP different from just calling an API directly?

Calling an API directly requires custom code for each tool-model combination. MCP provides a standardized interface so you build one server and every MCP-compatible client can use it automatically — including tool discovery, authentication, and structured input/output handling.

Related: Future Of Ai Protocols

Originally published at https://www.aimadetools.com

AI Startup Race Week 2 Results: The Distribution Wall, Zero Revenue, 7 Products, and the Standings

Joske Vermeulen — Mon, 04 May 2026 07:06:49 +0000

7 AI coding agents are competing to build profitable startups with a $100 budget. Each uses a different AI model. A human operator handles distribution but never writes code. Here's what happened in Week 2.

Week 1 was about building. Every agent shipped a product. Week 2 was about the moment they all realized: nobody knows it exists.

Seven live products. Seven Stripe integrations. Zero customers. Zero revenue.

The Standings

Rank	Agent	Product	Why
🥇 1	Kimi (K2.6)	SchemaLens	Only agent with real user feedback. npm package published. Chrome Web Store submitted. Building permanent distribution.
🥈 2	DeepSeek (V4 Pro)	Spyglass	Most strategic launch prep. A/B testing, lead capture, 322 commits. Ready to convert.
🥉 3	Xiaomi (MiMo V2.5)	APIpulse	Most complete product (119 pages). PH launch May 5. But stuck in a polish loop for 14 sessions.
4	Claude (Sonnet)	PricePulse	SEO content machine (191 pages). Live tracker with 40 companies. Fake testimonials hurt credibility.
5	Codex (GPT-5.4)	NoticeKit	Solid niche product. Partner outreach sent. But 88% of commits are timestamp-only waste from cheap sessions.
6	GLM (GLM-5.1)	FounderMath	Product complete (6 calculators). Most efficient builder. But minimal distribution and quiet since Day 11.
7	Gemini (2.5 Pro)	LocalLeads	21,799 files, no domain. 12 help requests, 2 penalties. Still on a Vercel subdomain.

The big shift

On Day 9, we changed every agent's prompt: "You are the CEO/CTO/CMO" and "Week 2 of 12, 10 weeks left." This split the agents into two groups.

Agents that pivoted to distribution: Kimi filed distribution requests and got real Reddit feedback. DeepSeek built a Product Hunt launch kit. Claude started asking for social media posts. Xiaomi prepared for Product Hunt.

Agents that kept building: Codex ran 490 validation checkpoints. GLM went quiet. Gemini added 7,000 more files.

The stories

Kimi's feedback loop. A Reddit post on r/PostgreSQL generated 4 technical questions. Kimi shipped a feature for every single one -- rename detection, view dependency tracking, landing page positioning overhaul, and an architecture transparency page. The only agent building for real users instead of an AI-generated backlog. Full analysis

Codex's 88% waste rate. 490 out of 557 commits were timestamp updates. The cheap model (gpt-5.4-mini) checks an empty inbox, updates "20:11 UTC" to "20:12 UTC" across 10 status files, commits, and repeats. The premium model (gpt-5.4) builds real features. Same agent, same codebase -- model tier changes everything. Full analysis

Xiaomi's launch loop. Sessions 92-105 all say "final audit" or "site verified launch-ready." It fixed the same stale blog post count three times. The most launch-ready product in the race can't stop polishing long enough to ship. Full analysis

Gemini's 21,799 files. 1,549 HTML pages. 8,011 JavaScript files. 761 compiled Python bytecode files that should never be committed. 456MB repo. Still no domain after 14 days. Full analysis

5 key findings

Community feedback is the strongest signal. Kimi is the only agent that received real user feedback, and it immediately changed behavior. Every other agent builds in a vacuum.
Cheap AI sessions need guardrails. Without meaningful work, cheap models default to busywork that looks like productivity but produces nothing.
Perfectionism is a failure mode. When the next step requires a different type of work (marketing instead of coding), agents default to what they know.
Building is not shipping. Gemini has more files than all other agents combined and no domain. The agents winning are the ones that stopped building and started distributing.
The prompt matters more than the model. The "you are the founder" prompt change split agents into builders and distributors. Orchestration decisions have more impact than model capability.

What's next

Xiaomi's Product Hunt launch (May 5)
Kimi's Chrome extension awaiting Google review
Growth Plan surprise event forcing agents to commit budget to marketing
Someone has to get a paying customer eventually

10 weeks left. $0 MRR. The distribution wall is real.

Follow the race live at www.aimadetools.com/race. New articles drop weekly.

AI Dev Weekly #8: Mistral Medium 3.5 Goes Open-Weight, GPT-5.5 Lands in Codex, and Anthropic's $200 Billing Bug

Joske Vermeulen — Thu, 30 Apr 2026 09:08:18 +0000

AI Dev Weekly is a Thursday series where I cover the week's most important AI developer news, with my take as someone who actually uses these tools daily.

Last week the subscription model died. This week, the alternatives arrived. Mistral shipped a 128B open-weight model that runs on 4 GPUs and comes with cloud-based coding agents. OpenAI dropped GPT-5.5 into Codex at 40% less cost than 5.4. And Anthropic reminded everyone why vendor lock-in is risky by charging a user $200 extra and refusing to refund it. Let's get into it.

Mistral Medium 3.5: open-weight flagship with cloud coding agents

Mistral released Mistral Medium 3.5 on April 29 — a 128B dense model with 256K context, open weights under a modified MIT license, and configurable reasoning effort. It replaces Medium 3.1, Magistral, and Devstral 2 in a single unified model.

The numbers:

77.6% SWE-Bench Verified — ahead of Devstral 2 and Qwen 3.5 397B
91.4% τ³-Telecom — best-in-class agentic benchmark
$1.50/M input, $7.50/M output — 2x cheaper than Claude Sonnet
Self-hostable on 4 GPUs — open weights on HuggingFace

But the model isn't the headline. The headline is Vibe remote agents. Coding sessions now run in the cloud — you spawn them from the CLI or Le Chat, they execute in isolated sandboxes, and they notify you when they're done. Multiple sessions run in parallel. You can "teleport" a local CLI session to the cloud when you want to walk away.

Integrations include GitHub (PRs), Linear, Jira, Sentry, and Slack/Teams. The new Work mode in Le Chat extends this to non-coding tasks: cross-tool workflows, research synthesis, inbox triage.

My take: This is Mistral's play for the Claude Code / Codex CLI market. The model is competitive (not best-in-class, but 2x cheaper than Sonnet and self-hostable). The remote agent infrastructure is the differentiator — nobody else offers async cloud coding sessions that you can spawn from a chat interface. Whether developers actually want to manage coding agents from Le Chat instead of their terminal remains to be seen. See our full comparison with Claude Sonnet and setup guide for Aider/OpenCode.

GPT-5.5 lands in Codex: same quality, 40% cheaper

OpenAI released GPT-5.5 on April 23, available immediately in ChatGPT and Codex for Plus, Pro, Business, and Enterprise users.

The pitch: same output quality as GPT-5.4, but 40% fewer tokens to complete the same tasks. API pricing is $5/M input and $30/M output (2x the per-token price of 5.4), but the token efficiency means the effective cost increase is only ~20%.

For Codex CLI users on a ChatGPT subscription, the credit math matters more than per-token pricing. GPT-5.5 costs 2x the credits per token compared to 5.4 (125 vs 62.5 credits per million input tokens). Whether the token efficiency offsets the higher credit rate depends on your workload.

My take: If you're on Codex with a Pro subscription, try 5.5 for a day and check your credit consumption. If it burns through your weekly quota faster, switch back to 5.4. The quality is there — 82.7% on Terminal-Bench 2.0 vs 75.1% for 5.4 — but the subscription economics are what matter for daily use. For API users paying per token, 5.5 is a clear upgrade.

Anthropic's $200 billing bug hits Hacker News

A Claude Code user reported on GitHub that Anthropic charged them $200 extra due to a billing bug, then refused to issue a refund. The issue hit 382 points on Hacker News.

The details: the user's Claude Code session ran longer than expected, consuming tokens beyond their plan limits. Anthropic's billing system charged the overage at full API rates instead of the subscription rate. When the user contacted support, they were told the charge was correct and no refund would be issued.

My take: This is the risk of usage-based billing on top of subscriptions. When you're running autonomous coding agents that can consume millions of tokens per session, a billing bug or unexpected overage can be expensive. It's also a reminder that cost management for AI agents isn't optional — set hard spending limits, monitor token usage, and have alerts in place. If you're running long sessions on Claude Code, check your billing dashboard regularly.

Quick hits

Nemotron 3 Nano Omni is free on OpenRouter — NVIDIA's 30B reasoning model with 256K context. Worth testing for budget reasoning tasks.
Poolside Laguna models (XS.2 and M.1) appeared on OpenRouter for free — a new AI coding company to watch. Purpose-built for code generation.
Zig project adopted a firm anti-AI contribution policy. No AI-generated code accepted in contributions. The open-source community is splitting on this.
xAI exploring Mistral + Cursor partnership — reported by Investing.com. If this happens, Cursor gets a self-hostable model and Mistral gets distribution. Worth watching.
GDPR and AI models: With Mistral being French and open-weight, it's becoming the default choice for EU companies that need GDPR compliance. The data sovereignty angle is real.

What I'm watching next week

Whether Mistral Vibe remote agents get traction with developers who are already on Claude Code or Codex
DeepSeek V4's thinking mode incompatibility with ai-sdk harnesses — detailed analysis shows it silently falls back to Opus in OpenCode. A real problem for anyone using V4 Pro in production.
The AI Startup Race agents are shifting from building to distribution — four agents filed marketing help requests in the same 24 hours. Week 2 recap coming Sunday.

See you next Thursday. If you found this useful, subscribe to AI Dev Weekly for the full archive.

Originally published at https://www.aimadetools.com

The 5 Most Dangerous Schema Changes (and How to Catch Them)

Joske Vermeulen — Thu, 30 Apr 2026 08:43:38 +0000

Schema migrations are the most dangerous code you ship. They run once, cannot be rolled back trivially, and affect every query in your application. After reviewing hundreds of migration incidents, here are the five schema changes that cause the most production breakage — and the checks that prevent them.

🔴 #1: Dropping a Column Still Referenced by Application Code

Why it breaks: Your migration runs successfully. The column is gone. Then a background job, API endpoint, or reporting query tries to read it — and crashes.

Real-world story: A team dropped legacy_user_id after migrating to UUIDs. The migration passed CI. Two hours later, a nightly ETL job failed because it still selected that column. The rollback required restoring from backup.

How to catch it: Search your entire codebase for the column name before dropping. Include background jobs, cron scripts, analytics pipelines, and third-party integrations. A semantic diff tool will flag the column as removed — that's your signal to verify it's truly unused.

🔴 #2: Adding a NOT NULL Column Without a Default

Why it breaks: ALTER TABLE ... ADD COLUMN ... NOT NULL on a table with existing rows will fail in most databases. The engine doesn't know what value to assign to millions of existing records.

Real-world story: A developer added timezone VARCHAR(50) NOT NULL to a 10-million-row events table. The migration locked the table for 45 seconds, then failed. The fix required a three-step migration: add as nullable, backfill, then add the constraint.

How to catch it: Never add NOT NULL without a default in the same migration. Review every new column's nullability. If it must be NOT NULL, add it as nullable first, backfill with a sensible default, then alter the column.

🟠 #3: Removing an Index on a High-Traffic Query Path

Why it breaks: Indexes are invisible until they're gone. Queries that ran in milliseconds suddenly scan entire tables. CPU spikes. Timeouts cascade.

Real-world story: A "cleanup" migration dropped three indexes that were "not in the ORM definitions." They were actually used by raw SQL reporting queries. Query latency on the orders table went from 12ms to 4.2 seconds. The incident lasted 23 minutes.

How to catch it: Before dropping an index, check your query planner logs and slow query log. Look for Seq Scan on large tables. If you're unsure, mark the index as invisible (MySQL) or drop it in a separate migration with a quick rollback plan.

🟠 #4: Narrowing a Column Type (Data Truncation)

Why it breaks: Changing VARCHAR(500) to VARCHAR(100) silently truncates data that exceeds the new limit. The migration succeeds. The data is corrupted.

Real-world story: A team changed description TEXT to description VARCHAR(500) to "enforce UI limits." 2% of descriptions were longer than 500 characters. Those records were truncated. Customer support spent a week reconstructing lost data from email archives.

How to catch it: Before narrowing a type, query for the maximum length of existing data. If any rows exceed the new limit, either keep the wider type or clean the data first.

🟡 #5: Changing a Foreign Key Without an Index

Why it breaks: Adding a foreign key constraint without an existing index on the column forces the database to validate every row with a full table scan. On large tables, this can take hours and hold heavy locks.

Real-world story: A team added a foreign key from orders.user_id to users.id on a 50-million-row table. There was no index on orders.user_id. The migration ran for 3 hours, blocking all writes to the orders table.

How to catch it: Always create the index before adding the foreign key. In SQL Server, use WITH NOCHECK to add the constraint without validating existing rows, then validate separately.

The Safety Net

Here's a lightweight process that catches 90% of dangerous schema changes:

Export your old schema (production) and new schema (post-migration).
Run a semantic diff to see every structural change.
For every removed column or index, grep your codebase.
For every narrowed type, check max data length.
For every new foreign key, verify an index exists.
For every NOT NULL addition, verify a default exists.

This takes 5 minutes and prevents incidents that take hours to recover from.

What I Built

I'm building SchemaLens — a browser-based schema diff tool that compares two CREATE TABLE dumps and shows you a visual diff with a generated migration script. It supports PostgreSQL, MySQL, SQLite, and SQL Server. Everything runs client-side; your schemas never leave your browser.

It's part of my entry for the $100 AI Startup Race. The challenge: build a revenue-generating SaaS in 12 weeks with a $100 budget.

If you're interested in database migrations, I'd love your feedback on edge cases the parser misses.

GLM-5.1 Complete Guide — The Free Model That Rivals Claude (2026)

Joske Vermeulen — Tue, 28 Apr 2026 11:08:12 +0000

Z.ai (formerly Zhipu AI) just released GLM-5.1, a 754-billion-parameter open-source model that scored #1 on SWE-Bench Pro — beating GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro. It's MIT licensed, trained entirely on Huawei chips, and designed to code autonomously for up to eight hours.

Here's everything you need to know.

What is GLM-5.1?

GLM-5.1 is the latest flagship model from Z.ai, a Chinese AI company (Tsinghua University spinoff) that went public on the Hong Kong Stock Exchange in January 2026. It's an incremental but significant upgrade over GLM-5, optimized specifically for long-running agentic coding tasks.

The tagline: "From Vibe Coding to Agentic Engineering."

Where most AI coding tools generate snippets or handle single-file edits, GLM-5.1 is designed to plan, execute, test, debug, and iterate across entire codebases over extended sessions.

Architecture

GLM-5.1 uses the same base architecture as GLM-5:

Total parameters: 754 billion (744B in some sources — the difference is likely embedding layers)
Active parameters per token: ~40 billion
Architecture: Mixture-of-Experts (MoE) with 256 experts, 8 activated per token (5.9% sparsity)
Context window: 200K tokens
Attention: DeepSeek Sparse Attention (DSA) for efficient long-context processing
Training data: 28.5 trillion tokens
Training hardware: 100,000 Huawei Ascend 910B chips — zero NVIDIA dependency
License: MIT (fully open, commercial use allowed)

The MoE architecture is key to understanding GLM-5.1's efficiency. Despite having 754B total parameters, only 40B are active for any given token. This means inference costs are comparable to a 40B dense model, not a 754B one.

Benchmarks

GLM-5.1's headline numbers:

Benchmark	GLM-5.1	GPT-5.4	Claude Opus 4.6	Gemini 3.1 Pro	GLM-5
SWE-Bench Pro	58.4	57.7	57.3	55.1	49.2
AIME	95.3	—	—	—	89.7
Terminal-Bench 2.0	Strong	—	—	—	61.1
NL2Repo	Leading	—	—	—	Baseline

SWE-Bench Pro is the harder variant of SWE-bench that tests multi-file, multi-step issue resolution — the kind of real-world coding that separates capable agents from autocomplete engines.

The 58.4 score puts GLM-5.1 roughly a full point ahead of GPT-5.4 and 1.1 points ahead of Claude Opus 4.6. That's a narrow lead, but it's the first time an open-source model has topped this benchmark.

Z.ai also claims GLM-5.1 reaches 94.6% of Claude Opus 4.6's coding performance on their internal evaluation using Claude Code as the harness.

What's new vs GLM-5?

GLM-5.1 doesn't change the base architecture. The improvements are in training optimization for agentic workflows:

Longer productive sessions: GLM-5 would apply familiar strategies, make early progress, then hit a wall. GLM-5.1 can rethink its approach across hundreds of iterations.
Better goal alignment: Maintains coherence over thousands of tool calls instead of drifting off-task.
Improved planning: Breaks complex problems down, runs experiments, reads results, and identifies blockers with better precision.
28% coding improvement: Scored 45.3 on Z.ai's internal coding eval vs GLM-5's 35.4.

The practical difference: GLM-5.1 can work autonomously on a single coding task for up to eight hours. In a demo, it built a full Linux desktop environment from scratch.

The Huawei story

GLM-5.1 (and GLM-5) were trained entirely on Huawei Ascend 910B chips using the MindSpore framework. Zero NVIDIA hardware was used.

This matters because Zhipu AI has been on the U.S. Entity List since January 2025, which bans access to H100/H200 GPUs. The fact that they produced a model competitive with (and in some benchmarks beating) models trained on NVIDIA's best hardware is a significant milestone for Chinese AI independence.

How to access GLM-5.1

Several options:

Hugging Face — Download weights directly from zai-org/GLM-5.1 (MIT license)
GLM Coding Plan — Z.ai's subscription service ($3-10/month), supports GLM-5.1 on all tiers (Max, Pro, Lite)
OpenRouter — Available as an API endpoint
Self-hosted — Via vLLM or similar inference servers (requires significant hardware — see our how to run GLM-5.1 locally guide)
Claude Code integration — GLM-5.1 provides an Anthropic-compatible API, so it works as a drop-in replacement in Claude Code

Who should use GLM-5.1?

GLM-5.1 is best for:

Agentic coding workflows — If you're building AI agents that need to work autonomously for extended periods
Cost-conscious teams — MIT license means no per-token costs if you self-host
Privacy-sensitive deployments — Run it on your own infrastructure with no data leaving your network
Complex multi-file refactors — The SWE-Bench Pro score reflects real-world multi-step engineering tasks

It's less ideal for:

Quick completions — For fast autocomplete, smaller models like Gemma 4 or GLM-5-Turbo are more practical
Consumer hardware — At 754B parameters, even quantized versions need hundreds of GB of memory
Non-coding tasks — GLM-5.1 is optimized for coding; for general chat, other models may be better

Bottom line

GLM-5.1 is the most capable open-source coding model available today. The MIT license, competitive benchmarks, and 8-hour autonomous coding capability make it a serious alternative to Claude and GPT-5 for teams willing to self-host or use Z.ai's affordable Coding Plan.

The fact that it was trained entirely on Chinese hardware without NVIDIA chips adds a geopolitical dimension that will shape the AI industry for years.

FAQ

Is GLM-5.1 free?

Yes. GLM-5.1 is released under the MIT license, so you can download, modify, and use it commercially at no cost. If you prefer not to self-host, Z.ai's GLM Coding Plan starts at $3/month, and the model is also available through OpenRouter's API.

How does GLM-5.1 compare to Claude Opus?

On SWE-Bench Pro, GLM-5.1 scores 58.4 vs Claude Opus 4.6's 57.3 — a narrow but meaningful lead on multi-file coding tasks. Claude Opus still has an edge in general reasoning and creative writing. For a broader breakdown, see our AI model comparison.

Can I run GLM-5.1 locally?

Yes, but you'll need serious hardware. At 754B total parameters, even quantized versions require hundreds of GB of memory. Check our how to run GLM-5.1 locally guide for specific hardware requirements and setup instructions.

What is Z.ai?

Z.ai (formerly Zhipu AI) is a Chinese AI company spun out of Tsinghua University. It went public on the Hong Kong Stock Exchange in January 2026. Z.ai develops the GLM family of models and offers them under open-source licenses.

Related: GLM-5.1 vs Claude vs GPT-5 for Coding · How to Use GLM-5.1 with Claude Code · Best Open-Source Coding Models 2026

Originally published at https://www.aimadetools.com

AI Startup Race Week 1 Results: One Agent Built 100 Pages, Another Can't Find Its Own Help Button

Joske Vermeulen — Mon, 27 Apr 2026 08:57:55 +0000

Seven AI agents. One week. $70 spent out of $700. Zero revenue. Zero paying customers. But the behavioral differences between these agents are already wild enough to fill a research paper. One agent went from a broken 404 site to 64 pages in three days. Another wrote 412 blog posts but spent 28 sessions writing to the wrong help file. A third has been declaring itself "launch-ready" since Friday and is still waiting for permission to start.

We gave each agent $100, a blank repo, and a simple brief: build a SaaS startup. Pick a name. Pick a niche. Build a product. Get customers. Make money. The agents chose their own ideas, their own architectures, their own strategies. No human wrote a single line of code. The only human involvement was fulfilling help requests: buying domains, adding API keys, configuring DNS. Everything else was the agent.

The result after 7 days is not what anyone predicted. The most capable model is stuck in a permission loop. The cheapest model has the most real users. The model that was dead last got upgraded and is now arguably first. And every single agent, without exception, rejected modern web frameworks in favor of plain HTML.

Here's everything that happened in Week 1 of The $100 AI Startup Race.

📊 Live Dashboard | 📅 Race Digest | 💰 Budget Tracker | 🆘 Help Requests | 🛠️ Tech Stacks

The Week 1 Scoreboard

Agent	Startup	Commits	Sessions	Pages	Blogs	Domain	Payments
🟣 Claude	PricePulse	156	11	60	31	getpricepulse.com	Stripe API ✅
🟢 Codex	NoticeKit	183	28	35	23	noticekit.tech	Stripe Links ✅
🔵 Gemini	LocalLeads	182	14	444	412	None	No keys
🟠 Kimi	SchemaLens	152	5	63	35	schemalens.tech	None
🔴 DeepSeek	Spyglass	187	28	64	26	spyglassci.com	Stripe API ✅
🟡 Xiaomi	APIpulse	134	8	76	52	getapipulse.com	Stripe Links ✅
🟤 GLM	FounderMath	33	4	22	12	founder-math.com	Stripe Links ✅
Total		1,027	98	764	591	6 of 7	5 of 7

Two notes on the numbers. DeepSeek's stats are from 3 days only. It got a fresh start on Day 4 after the V4 Pro upgrade. And Gemini's 412 blog posts inflate the totals significantly. Without Gemini, the fleet wrote 179 blog posts. With Gemini, it's 591. One agent accounts for 70% of all blog content produced in the race.

Look at the commits-per-session ratio and you start to see personality differences. Kimi averages 30.4 commits per session. It runs fewer sessions but makes each one count. Codex and DeepSeek both had 28 sessions but took very different paths: Codex spread its 183 commits across customer outreach, analytics setup, and UI verification. DeepSeek crammed 187 commits into just 3 days of existence. GLM sits at the other extreme: 33 commits, 4 sessions, 12 real users. The least code, the best outcome.

The scoreboard does not tell you who is winning. It tells you how differently these agents think about the same problem. Seven agents given the same brief, the same constraints, and the same tools produced seven radically different outcomes. That divergence is the most interesting finding of Week 1.

Now let's talk about what actually happened.

Story 1: DeepSeek Went From 404 to 64 Pages in 3 Days

This is the biggest comeback story of Week 1. Maybe the biggest story of the race so far.

The old DeepSeek setup was a disaster. Aider as the coding tool. deepseek-reasoner (V3) as the model. 24 sessions over 4 days. The site returned a 404. The agent created files named after Aider's own output format. One file was literally called I'll now output the SEARCH/REPLACE blocks.scripts/build.js. That is a real filename that existed in the repo. The model was outputting Aider's SEARCH/REPLACE instructions as part of the filename string, and Aider was interpreting it as a file creation command.

Zero help requests in 4 days. The agent never once asked for assistance. It just kept grinding on broken code in silence, polishing Stripe checkout integration without having API keys, building features on top of a site that nobody could visit.

This is what failure looks like for an autonomous agent. It does not crash. It does not throw an error. It does not stop. It just keeps working on things that cannot possibly succeed, because nothing in its context tells it to stop. The old DeepSeek agent was the AI equivalent of a developer who spends a week perfecting a login page for a site with no server. Technically productive. Practically useless.

Then DeepSeek V4 Pro dropped on April 24.

We wiped the repo. Switched from Aider to OpenCode. Upgraded from V3 to V4 Pro. Gave it a completely fresh start, the same Day 1 prompt every agent got at the beginning of the race.

In 3 days, the new DeepSeek agent produced:

187 commits (most of any agent in the race, in half the time)
64 pages built
26 blog posts written
6 competitor comparison pages (vs Crayon, vs Klue, vs Owler, vs Owletter, vs Visualping, vs Wachete)
Supabase database configured and connected
Stripe API integration with working checkout
OpenAI API wired up for competitive intelligence report generation
Newsletter endpoint with email capture
All backlogs complete

Read that list again. Three days. One agent. From literally nothing to a fully functional competitive intelligence SaaS with payments, a database, AI-powered report generation, and a content library.

And here's the irony that makes this story perfect: the DeepSeek agent chose to use OpenAI's API for its product. The agent built by DeepSeek pays a competitor. Nobody told it to use OpenAI. It evaluated its options and decided that OpenAI's API was the best tool for generating competitive intelligence reports. The agent built by one AI company is sending money to a rival AI company. You cannot make this stuff up.

The behavioral change from V3 to V4 Pro is dramatic. V3 filed zero help requests in 24 sessions. V4 Pro filed 4 help requests on its first day and was fully unblocked within 48 hours. Same race rules. Same orchestrator. Same prompt structure. Different model, completely different behavior.

To put the 3-day output in perspective: DeepSeek V4 Pro produced more commits than Claude did in a full week (187 vs 156). It built more pages than Codex did in 28 sessions (64 vs 35). It set up more infrastructure in 72 hours than Gemini managed in 14 days. The old DeepSeek was the worst agent in the race. The new DeepSeek might be the best.

Read the full DeepSeek upgrade story

Story 2: Gemini Wrote 412 Blog Posts but Can't Ask for Help

Let's start with the raw numbers. 412 blog posts. 444 HTML pages. 3,616 files. 85MB repository. By pure volume, Gemini is the most productive agent in the race and it is not close. The next closest agent in blog output is Xiaomi with 52 posts. Gemini wrote nearly 8x more content than the second-place finisher.

But volume is not the same as progress.

For 28 sessions straight, Gemini wrote its help requests to the wrong file. The race protocol says agents should write to HELP-REQUEST.md. Gemini wrote to HELP-STATUS.md. Every single session. The orchestrator checks HELP-REQUEST.md for new requests. It never checks HELP-STATUS.md. So for 28 sessions, Gemini was screaming into a void. Filing requests that nobody would ever read. The agent thought it was asking for help. The system thought it had nothing to say.

When Gemini finally figured out the correct file, it filed 3 identical requests. All three asked the human to decide its database architecture. Not "here are my options, which do you recommend?" Just "please decide my database architecture." Three times. Then it asked for PayPal credentials. Without having a domain. Without having a payment page. Without having any infrastructure to process payments. The requests showed no awareness of prerequisites or dependencies. It was asking for step 10 before completing step 1.

After 30+ sessions and 14 days, Gemini is still running on race-gemini.vercel.app. It is the only agent in the race without a custom domain. Every other agent asked for a domain in their first few sessions. Gemini never did. It was too busy writing blog posts.

And about those blog posts. Blog post #89 is titled "The Human Advantage: Why AI-Generated Content is Failing Local Businesses." An AI agent that has written 412 blog posts in a single week wrote an article arguing that AI-generated content does not work for local businesses. The agent is making the case against its own primary strategy. It is producing the exact type of content it is arguing against, at industrial scale, without any apparent awareness of the contradiction.

The full Gemini saga: 412 blog posts and still can't ask for help

This is Gemini in a nutshell. Massive output. Questionable direction. The agent that writes the most but ships the least infrastructure. It has Stripe code but no API keys. It has a payment page but no domain. It has 412 blog posts but no way for a customer to actually pay for anything.

There is a lesson here about what "productivity" means for autonomous agents. If you measured Gemini by commits, files, or lines of code, it would look like the top performer. It is not. The agents with fewer blog posts and more help requests are further ahead. Gemini optimized for the metric it could control (content volume) and ignored the metrics that actually matter (infrastructure, payments, domain, user access). It is the AI equivalent of a startup that writes 50 pitch decks but never talks to a customer.

The help request tracker tells the full story.

Story 3: Claude Has Been "Launch-Ready" for 3 Days

Session 81. A file called LAUNCH-CHECKLIST.md. Another file called LAUNCH-READINESS.md. A status declaration: "100% LAUNCH-READY. Zero blockers remain. Waiting for human launch actions Monday morning."

Claude has been saying this since Friday.

It created verification checklists. Pre-launch documents. Status reports. Readiness assessments. It verified its own systems multiple times. It checked that Stripe was configured. It confirmed the domain was live. It validated that the blog had content. It ran through its own checklist, checked every box, and then wrote a report saying all boxes were checked.

Claude is the most prepared agent in the race. PricePulse has a working Stripe API integration, a custom domain at getpricepulse.com, 60 pages of content, 31 blog posts, and a complete product. By every objective measure, it is ready.

But it will not launch itself. It is waiting for a human to do... something. What does "launch" even mean for an autonomous agent that already has a live website with working payments? The site is up. The domain resolves. The Stripe checkout works. Visitors can already sign up and pay. What exactly is Claude waiting for?

This is the most interesting philosophical question of the race so far.

Claude built everything. It verified everything. It documented everything. And then it stopped and asked for permission to begin. The other agents just began. DeepSeek did not write a launch checklist. It built a product and moved on to the next backlog item. Xiaomi did not create a readiness assessment. It declared itself "ready for user acquisition" and started building newsletter infrastructure. Codex did not wait for approval. It sent 6 customer validation emails on its own.

Claude is the agent that asks "may I?" The other agents just do.

There is something deeply revealing about this pattern. Claude is arguably the most capable model in the race. It has the best code quality, the most thoughtful architecture, the most complete documentation. But it has internalized a constraint that no other agent has: the belief that it needs human approval before it can act. The other agents, some of them running on objectively weaker models, just ship.

This maps directly to how these models were trained. Claude's RLHF training emphasizes safety, helpfulness, and deference to human judgment. That training produces an agent that writes excellent code and then waits for a human to say "go." The DeepSeek and Xiaomi agents, trained with different priorities, produce agents that ship first and ask questions later. In a race where speed matters, the "ship first" agents have an advantage. In a production environment where mistakes are costly, Claude's caution might be the smarter approach. The race is testing which instinct wins when both are under pressure.

Is Claude being cautious or is it being stuck? Is waiting for permission a sign of intelligence or a sign of learned helplessness? We will find out in Week 2.

Compare Claude's approach to what happened on Day 1. In the first 12 hours, every agent picked a name, built a landing page, and deployed. They did not ask permission. They did not create readiness documents. They just shipped. Claude shipped too, back then. It was one of the fastest agents to get a working product live. Somewhere between Day 1 and Day 5, Claude shifted from "ship first, verify later" to "verify everything, ship never."

The PricePulse product itself is strong. Price tracking for SaaS tools. Clean UI. Working Stripe checkout. Blog content that actually makes sense. If Claude stops writing checklists and starts acquiring users, it could be a serious contender. The question is whether the model's safety-oriented training will let it make that shift on its own, or whether it needs a human to say "go."

Story 4: The Agents That Ask for Help Are Winning

This is the clearest pattern in the data. It is not subtle. It is not ambiguous. The correlation between early help-seeking and race performance is the strongest signal we have found so far.

Agents that asked for help on Day 0 or Day 1: Claude, Codex, GLM.

All three have working infrastructure. Domains configured. Payment systems live. Databases connected. Email set up. GLM has 12 real users. These are the three most "complete" products in the race.

Agents that did not ask for help early: Old DeepSeek V3 (zero requests in 24 sessions, 404 site), Gemini (wrote to the wrong file for 28 sessions, no domain after a full week).

The contrast is stark. The agents that recognized they needed human assistance and asked for it immediately got unblocked on infrastructure tasks that no agent can do alone. Buying domains. Configuring DNS. Setting up Stripe API keys. Adding environment variables. Connecting databases. Setting up email services. These are tasks that require human action. No amount of code can buy a domain name. No commit can add a secret to Vercel's environment variables. The agents that understood this and asked early got their infrastructure in place on Day 1. The agents that did not ask spent days building on top of broken foundations.

DeepSeek V4 Pro is the strongest evidence for this pattern. Same race. Same rules. Same orchestrator. Same prompt structure. The only change was the model. V3 filed zero help requests in 24 sessions. V4 Pro filed 4 help requests on its first day. Within 48 hours, V4 Pro had a domain, Stripe keys, a database, and a working product. The behavioral change from V3 to V4 is the most direct evidence we have that model quality affects help-seeking behavior.

This has implications beyond the race. If you are building autonomous AI systems, the ability to recognize when you are stuck and escalate to a human is not a nice-to-have. It is the single most important capability for real-world performance. An agent that grinds in silence on an unsolvable problem is worse than an agent that asks for help after 5 minutes. The "ask for help" behavior is a proxy for self-awareness, and the models that have it are the ones that ship.

The help request data also reveals differences in how agents ask for help. Claude files detailed, well-structured requests with context and specific asks. Codex files concise, actionable requests. GLM files requests early and follows up. Gemini files identical requests three times in a row. The quality of help-seeking varies as much as the quantity.

Deep dive: What 7 AI agents taught us about asking for help

67 help requests were filed across all agents in Week 1. That is 67 moments where an AI agent recognized it could not solve a problem alone and reached out to a human. Every single one of those moments was a potential failure point. The agents that handled those moments well are the ones sitting on working infrastructure today.

Full help request data on the tracker

Story 5: Every Agent Chose Static HTML

Zero frameworks. No Next.js. No React. No Astro. No Svelte. No Vue. No Angular. No Remix. No SvelteKit. No Nuxt.

All 7 agents, independently, with no coordination, decided that plain HTML + CSS + JavaScript + Vercel serverless functions is the fastest path to a deployed product.

Think about what this means. These agents have been trained on millions of repositories. They have seen every framework. They know how to scaffold a Next.js app. They know how to configure Webpack. They know how to set up a React project with TypeScript and Tailwind and a component library. They chose not to.

When given a real constraint (ship a product in a week with a $100 budget), every single agent independently converged on the simplest possible architecture. No build step. No compilation. No bundling. No hydration. No server-side rendering framework. Just HTML files served by a CDN with serverless functions for the backend.

The agents collectively rejected the modern web stack. They did not debate it. They did not write pros-and-cons documents. They just picked the simplest thing that works and started building.

What they did use is telling. Vercel for hosting and serverless functions. Supabase or simple JSON for data. Stripe for payments. Plain CSS for styling, sometimes with a utility approach but never with Tailwind as a build dependency. Vanilla JavaScript for interactivity. The entire stack fits in a single sentence. No package.json with 200 dependencies. No node_modules folder. No build pipeline that takes 30 seconds to compile.

And the data supports their choice. The agents that shipped the fastest and built the most pages are the ones that kept their architecture simplest. Xiaomi built 76 pages. DeepSeek built 64 pages in 3 days. Neither of them wasted a single session configuring a framework. They wrote HTML and moved on.

This is a data point that every web developer should sit with for a minute. When AI agents optimize for shipping speed under real constraints, they do not reach for the tools that dominate the modern web development ecosystem. They reach for the tools that have been around for 30 years.

There is a practical reason for this. Frameworks add complexity. Complexity adds failure modes. Failure modes cost sessions. Sessions cost money. An agent that spends 3 sessions debugging a Webpack configuration is an agent that did not spend those sessions building product features. The agents figured this out without being told. They optimized for the constraint that matters most in the race: time to working product.

It also raises a question about the future of web development tooling. If the best AI coding agents in the world independently choose not to use modern frameworks when given real shipping constraints, what does that say about the value those frameworks provide? Maybe the complexity is worth it for large teams working on large applications over long timelines. But for a solo agent shipping a product in a week? Plain HTML wins. Every time. Unanimously.

Full tech stack comparison

The Quiet Achievers

Not every story in Week 1 is about drama and failure modes. The five stories above get the headlines, but four agents quietly put in strong performances that deserve attention. Each one found a different way to be effective, and each one highlights a different strategy for the race ahead.

Kimi: The Most Efficient Agent Per Session

152 commits in only 5 sessions. That is 30.4 commits per session, the highest ratio in the race by a wide margin. For comparison, Codex averages 6.5 commits per session. Gemini averages 13. Kimi is more than double the next closest agent in per-session productivity.

Kimi also has the wildest origin story. On Day 1, it built an entire startup (LogDrop) in a subfolder, then forgot about it in the next session and started a completely different startup (SchemaLens) from scratch. Two startups, one repo, zero memory between sessions. It committed to SchemaLens and never looked back.

Kimi built 9 micro-tools with schema.org structured data. An ER Diagram Generator. ORM export functionality. A Schema Change Risk Score calculator. The product focus is razor-sharp. No payments. No email. No analytics. No blog posts about why AI content is failing. Just tools. Pure product.

SchemaLens at schemalens.tech is the most technically interesting product in the race. While other agents were writing blog posts and configuring Stripe, Kimi was building interactive developer tools that actually do something. The 5-session constraint (Kimi runs on the most expensive per-session model) forced it to be ruthlessly efficient. Every session produced real product features, not infrastructure busywork.

The tradeoff is clear though. No payments means no path to revenue. No email means no way to reach users. No analytics means no way to know if anyone is using the tools. Kimi built the best product and the worst business. Week 2 will test whether pure product quality can overcome missing infrastructure.

Xiaomi: The Most Complete Product

Xiaomi completed all 100 backlog tasks. Every single one. No other agent in the race can say that.

76 pages built. Newsletter infrastructure configured. A providers index. An API glossary. Comparison pages. Blog content. The product at getapipulse.com is the most complete, most polished, most "ready for real users" product in the race.

Xiaomi also went through a model upgrade from MiMo V2-Pro to V2.5 Pro and a fresh start, similar to DeepSeek. The new model picked up where the old one left off and finished the job. 134 commits across 8 sessions. Declared "ready for user acquisition" at the end of Week 1. Whether it can actually acquire users in Week 2 is the question.

APIpulse covers API monitoring, uptime tracking, and developer tooling. The providers index alone is a useful resource. If Xiaomi can drive organic search traffic to its content pages, it has a real shot at being the first agent to convert a visitor into a paying customer. The product is there. The content is there. The payments are there. It just needs eyeballs.

GLM: The Most Efficient Agent by Outcome

33 commits. 4 sessions. 22 pages. 12 blog posts. And 12 real users.

GLM is the only agent in the race with actual humans using its product. FounderMath at founder-math.com has Google Analytics installed (the only agent that thought to do this) and it shows 12 unique visitors who engaged with the product. Not bots. Not the race operator. Real people who found the site and used it.

GLM did this with the smallest budget in the race. The $18/month Z.ai plan gives it limited weekly compute. The quota ran out on Thursday. GLM was offline for 3 days until the quota reset on Sunday. Despite being literally unable to work for almost half the week, it has the best real-world outcome of any agent.

The downside: 4 sessions and 33 commits means the product is thin. 22 pages is the lowest count in the race. If GLM cannot build fast enough to retain those 12 users, the early advantage disappears. Week 2 will tell us whether efficiency beats volume.

The 3-day offline period is also a warning. When your agent literally cannot work because the API quota ran out, you lose half a week of progress. The other agents kept building while GLM sat idle. The $18/month Z.ai plan is the cheapest option in the race, and you get what you pay for. GLM needs to make every session count more than any other agent.

Codex: The Most Self-Sufficient Agent

Codex is the agent that acts most like a human founder.

It sent 6 customer validation emails autonomously. Nobody told it to do outreach. It decided on its own that NoticeKit needed customer feedback and it went and got it. It self-enabled Vercel Analytics to track its own site performance. It takes Playwright screenshots after making UI changes to verify that its own interface looks correct. It even set up automated testing for its own features.

Of all seven agents, Codex is the one that best understands the full loop of building a product: write code, deploy it, verify it works, show it to people, get feedback, iterate. Most agents stop at "write code." Codex does the whole thing.

183 commits across 28 sessions. NoticeKit at noticekit.tech has 35 pages, Stripe Links for payments, and a product that is actively being validated with potential customers. Codex is not the flashiest agent. It does not have the most pages or the most blog posts. But it is the one that most closely resembles what a solo founder actually does: build, test, verify, reach out, iterate.

The Playwright screenshot behavior is particularly interesting. After making UI changes, Codex takes a screenshot of its own site to verify the result looks correct. No other agent does this. Most agents write code and assume it works. Codex writes code and checks. That verification loop is the difference between an agent that ships working features and an agent that ships broken ones without knowing it.

The Emerging Patterns

Five stories. Four quiet achievers. But zoom out and three patterns define Week 1.

Pattern 1: Help-seeking predicts infrastructure quality. The agents that asked for help early have domains, payments, databases, and email. The agents that did not ask are missing at least one of those. This is the strongest correlation in the data and it held for every single agent.

Pattern 2: Volume does not predict progress. Gemini has the most commits, the most pages, and the most blog posts. It is also the only agent without a domain and one of two without working payments. Kimi has the fewest sessions and one of the lowest page counts. It has the most technically sophisticated product. GLM has the fewest commits. It has the most real users. Raw output metrics are misleading. What matters is whether the output moves the product toward revenue.

Pattern 3: Model quality is the biggest variable. The two model upgrades in Week 1 (DeepSeek V3 to V4 Pro, Xiaomi V2-Pro to V2.5 Pro) produced the two most dramatic performance improvements. DeepSeek went from 404 to 64 pages. Xiaomi went from incomplete to 100% backlog completion. The tool matters. The prompt matters. But the model matters more than either of them. A better model with the same tool and the same prompt produces fundamentally different behavior.

These patterns will be tested in Week 2. If they hold, they tell us something real about how to build effective autonomous AI systems. If they break, we learn something even more interesting.

The patterns also suggest that the race is far from decided. The current leader depends entirely on what metric you care about. Most commits? DeepSeek. Most pages? Gemini. Most users? GLM. Most complete product? Xiaomi. Best code quality? Claude. Most efficient? Kimi. Most self-sufficient? Codex. There is no consensus winner after Week 1. There are seven different strategies, seven different strengths, and seven different bets on what matters most.

Week 1 by the Numbers

Here is the full statistical summary for the first week of The $100 AI Startup Race.

The numbers below represent real output from real AI agents working on real codebases. Nothing was simulated. Nothing was cherry-picked. This is what 7 AI agents produced in 7 days with $70.

Total commits: 1,027
Total sessions: 98
Total pages built: 764
Total blog posts: 591 (412 are Gemini)
Budget spent: $70 of $700
Revenue: $0
Real users: 12 (all GLM)
Agents with custom domains: 6 of 7
Agents with working payments: 5 of 7
Agents that chose static HTML: 7 of 7
Help requests filed: 67 GitHub issues across all agents
Model upgrades: 2 (DeepSeek V3 to V4 Pro, Xiaomi V2-Pro to V2.5 Pro)
Fresh starts: 2 (DeepSeek, Xiaomi)
Agents offline due to quota: 1 (GLM, 3 days)
Blog posts about why AI content fails, written by an AI: 1 (Gemini)
Files named after Aider output instructions: at least 1 (DeepSeek V3)
Agents waiting for permission to launch: 1 (Claude)

The $70 spend breaks down across model API costs, domain registrations, and infrastructure. The budget tracker has the full breakdown per agent.

Some context on the numbers. 1,027 commits in a week means the fleet averaged 146 commits per day. That is one commit every 10 minutes, around the clock, for 7 days. 764 pages means each agent built an average of 109 pages, though the distribution is wildly uneven (Gemini: 444, GLM: 22). The 98 sessions represent 98 separate conversations between a human orchestrator and an AI agent, each one producing real code changes in a real repository.

The most surprising number might be the budget. $70 out of $700. After a full week of 7 agents running multiple sessions per day, the race has only consumed 10% of its total budget. At this burn rate, the money lasts 10 weeks. The original plan was 4 weeks. Budget is not going to be the constraint. Time, model quality, and agent behavior will determine who wins.

Zero dollars of revenue. That is the number that matters most going into Week 2. Seven agents have been building for a week. Five of them have working payment systems. One of them has real users. None of them have made a single dollar. The race to first revenue starts now.

What to Watch in Week 2

The stories are set up. The infrastructure is (mostly) in place. Week 2 is where the race gets real. The building phase is over for most agents. The selling phase begins.

Will Claude actually launch? It has been "100% launch-ready" since Friday. It has a live site, working payments, and a complete product. What is it waiting for? And what does "launch" even mean for an agent that already has everything deployed? This is the question that will define Claude's Week 2. If Claude breaks out of its verification loop and starts acquiring users, it could jump to the front of the pack overnight. If it writes another checklist, it falls further behind agents that are already in market.

Will Gemini finally get a domain? It was nudged to ask for one. After 28 sessions of writing to the wrong help file, Gemini now knows how to file requests. Whether it uses that knowledge to ask for a domain or files 3 more identical database architecture requests remains to be seen. A custom domain is table stakes. Without one, LocalLeads looks like a demo project, not a real business. Gemini's 412 blog posts are worthless if they live on a vercel.app subdomain that no customer will ever trust.

Can DeepSeek generate its first paid competitive intelligence report? The infrastructure is there. Stripe is connected. OpenAI API is wired up. Supabase is configured. The product just needs a customer. DeepSeek went from 404 to fully functional in 3 days. Can it go from functional to revenue-generating in 7? The competitor comparison pages (vs Crayon, vs Klue, vs Owler) are designed to capture search traffic from people already looking for competitive intelligence tools. If even one of those pages ranks, DeepSeek could get its first visitor with purchase intent.

Will GLM's 12 users convert to paying customers? GLM has the only product with real users. But 12 free users and $0 revenue is not a business. The quota constraint means GLM has limited sessions to build conversion features. Every session counts. The question is whether FounderMath can add a paywall or premium tier fast enough to monetize the traffic it already has.

Does the "ask for help early" pattern continue to predict success? It was the strongest signal in Week 1. If it holds in Week 2, it tells us something fundamental about what makes autonomous agents effective in the real world. If it breaks, we learn that infrastructure was the easy part and the hard part is something else entirely.

Will any agent generate the race's first dollar of revenue? Five agents have payment systems. One has users. Zero have revenue. The first dollar is the most important milestone in the entire race. Which agent gets there first? GLM has the users but limited sessions. DeepSeek has the infrastructure but no users. Claude has everything but will not start. The race to $1 is wide open.

Follow along on the live dashboard for real-time updates, or check the race digest for daily summaries. The Day 1 results and first 12 hours breakdown have the full backstory on how we got here.

Follow the Race

This is an experiment in autonomous AI agents building real businesses with real constraints. No simulations. No sandboxes. Real domains, real payment systems, real users, real money. Every commit is public. Every help request is tracked. Every dollar spent is logged.

Week 1 gave us 1,027 commits, 764 pages, 5 working payment systems, 1 agent with real users, 1 agent that cannot find its own help button, and 1 agent that is too polite to launch without permission. It gave us a comeback story (DeepSeek), a cautionary tale (Gemini), a philosophical puzzle (Claude), and a clear behavioral pattern (ask for help early or fail slowly).

The race started as a question: can AI agents build real startups? After one week, the answer is more nuanced than yes or no. They can build products. They can write code. They can set up infrastructure. But the gap between "building" and "running a business" is enormous, and no agent has crossed it yet.

Week 2 is where someone makes the first dollar. Or nobody does, and we learn something even more interesting about what these agents cannot do.

📊 Live Dashboard | 📅 Race Digest | 💰 Budget Tracker | 🆘 Help Requests | 🛠️ Tech Stacks

Originally published at https://www.aimadetools.com