DEV Community: David Odukoya

Spec-Driven Development for AI App Building: A Practical Start-to-Ship Workflow

David Odukoya — Fri, 09 Oct 2026 13:59:43 +0000

Originally published at zalcro.ai/guides/spec-driven-development

Introduction: Moving Beyond the Prompt

Software engineering is shifting. We are moving from human-written code assisted by autocomplete to Agentic Software Engineering (ASE). Agents powered by models like Claude 3.5 Sonnet or OpenAI Codex no longer just assist. Inside tools like Cursor, Claude Code, and Windsurf, they keep their own thread of control. They navigate repositories, run terminal commands, and build multi-file implementations.

The early experience is usually very productive, whether you are a non-technical founder using a conversational interface (a vibe coder) or a developer orchestrating terminal commands (an agentic builder). You describe an idea and an app appears on your screen. Then you scale past isolated snippets toward production software and hit the AI Productivity Paradox.

Generative AI dramatically speeds up how fast you can produce lines of code. But empirical telemetry shows that unstructured, prompt-first development often degrades team-level throughput, delivery stability, and long-term maintainability [arXiv, 2026].

This guide lays out a practical workflow for Spec-Driven Development (SDD), the methodology built to solve that paradox. SDD adds a rigorous planning layer before you build, so your AI tools work inside strict architectural guardrails. Whether you use Lovable, Bolt.new, Cursor, or Claude Code, it turns unpredictable conversational agents into deterministic software compilers.

A modern software architecture pipeline

Part 1: Why Prompt-First Development Breaks Down

If you build without a clear plan, the AI has to infer your architecture as it goes. That works for simple prototypes. It breaks down as the app grows.

The Illusion of Speed and Delivery Instability

The DevOps Research and Assessment (DORA) reports for 2024 and 2025 captured the tension of AI development at scale. Across a survey of nearly 5,000 technology professionals, 90% of organizations reported using AI tools and noted real gains in individual task completion [Google Cloud Blog, 2025]. Yet using these tools correlated directly and negatively with software delivery stability [DORA.dev, 2024].

AI amplifies whatever workflow you already have. If you speed up code generation without also improving architectural planning and validation, you can easily overwhelm your review process downstream. The result is a rising "rework rate," the volume of unplanned deployments needed just to fix user-visible bugs introduced by rapid development cycles [DORA.dev, 2024].

The Erosion of Codebase Hygiene

GitClear's longitudinal analysis documents the structural damage from unstructured AI generation. Researchers examined over 211 million lines of code written between 2020 and 2026 to track the habits that keep software healthy over time [GitClear, 2026]. They found codebase hygiene eroding in step with the rise of AI assistants.

Software Quality Metric	Pre-AI Baseline (2021/2022)	AI Era Measurement (2024–2026)	Impact / Implication
Code Churn	Baseline	+15% to +39.2% increase	More code gets reverted or heavily modified within two weeks of being written, which points to flawed first drafts.
Refactored ("Moved") Code	21% of all code changes	3.8% of all code changes	Developers have stopped steadily improving existing architecture, which leaves rigid, legacy-burdened systems.
Code Block Duplication	40.3 blocks per million lines	73.0 blocks per million lines (+81%)	Duplicated logic taxes future maintainers every time it needs to change, and it violates DRY (Don't Repeat Yourself).
Function Connectivity	343 method calls per 1k lines	223 method calls per 1k lines (-35%)	New code increasingly sits in isolated silos, which suggests reinvention instead of real integration.
Legacy Code Maintenance	1.7% of changes touched legacy	0.46% of changes touch legacy (-74%)	Older parts of the codebase sit frozen and ignored until they fail.

Prompt-first development also introduces security regressions. Industry studies indicate that 27% to 40% of AI-generated code contains exploitable vulnerabilities, such as SQL injection, path traversal, or insecure authentication logic [Preprints.org, 2026]. The models' coding ability is not the problem. The problem is missing architectural context. With unstructured prompting, the AI writes code in a vacuum. It makes silent assumptions about database schemas, state management, and authentication boundaries because you never defined them [mariano-aguero, 2026].

How Does Context Rot Trigger the 80% Cliff?

If you have built an app with AI, you have probably hit the 80% cliff. You get a working prototype fast, covering about 80% of the visual scope. Then you add cross-cutting concerns, like connecting user authentication to a relational database schema, and the isolated components start to break. Fixing a bug in the database layer causes a regression in the UI.

The technical cause is "context rot" [Josh Owens, 2026]. Large language models spread attention across a finite context window. Every token you add to the workspace, whether it's a prompt, a reasoning trace, a file read, or an error stack trace, uses up that attention budget [Anthropic, 2026].

As you debug, the context window fills with a tangle of failed attempts and shifting instructions. When it nears capacity, AI environments charge a "compaction tax" [Josh Owens, 2026]. The system automatically summarizes the chat history to free up space. That summary flattens nuance. It blends discarded approaches with new directives, and the agent loses the thread of your original intent [Josh Owens, 2026].

Researchers at METR quantified this limit with the "50%-task-completion time horizon." The metric is the longest software engineering task an AI model can finish with a 50% success rate [METR, 2026]. On long-horizon, repository-scale benchmarks, models that dominated synthetic function-level tests repeatedly failed to speed up real-world workflows [METR, 2026]. Developers using these agents on complex tasks took 19% longer to finish them, even though they felt more productive [arXiv, 2026].

What Academic Research Says About Architectural Limits

Research on long-horizon, repository-scale benchmarks backs up what practitioners see. Foundation models do well on isolated, function-level tests like HumanEval, but they degrade when they have to coordinate logic across multiple files or evolve an existing architecture [SWE-bench PRO, OpenReview, 2026].

METR's "50%-task-completion time horizon" is the longest software engineering task an AI model can complete with a 50% success rate [METR, 2026]. On complex, real-world repositories, developers using these agents took 19% longer to finish tasks, even though they believed they were working 20% faster [arXiv, 2026].

Repository-scale benchmarks isolate specific failure modes. The RepoReasoner benchmark found that LLMs have high precision but critically low recall in multi-hop dependency tracing. They can find individual components but cannot reliably reconstruct how those components interact [arXiv, 2026]. When an agent can't map component relationships, architectural erosion follows. Studies of AI-synthesized microservices found an 80% architectural violation rate, with models bypassing established interfaces and creating circular dependencies to get a local fix working [arXiv, 2026].

The pattern holds across benchmarks: LLMs are strong syntax generators but unreliable system architects. Expecting an agent to infer a scalable architecture from a conversational prompt asks for something the technology can't do.

The Economic Cost of the Agent Loop

Context rot shows up directly as token waste. Agent pricing can hide the true cost of an implementation, because the expense comes from the continuous agent loop, not from any single prompt [YouTube, 2026].

In the agent loop, the AI reads context, generates code, watches the compiler fail, analyzes the error, and tries again [YouTube, 2026]. Research on agentic coding tasks shows these workloads can consume up to 1,000 times more tokens than standard chat interactions [Falconer, 2026]. When budgeting software development costs, engineers should apply a 1.7x to 2.0x multiplier to cover the overhead of automated retries, system prompt injection, and context repopulation [Iternal, 2026].

An agent stuck in a debugging loop, with no stable specification, will guess at architectural fixes and burn through your compute credits. A task that should cost cents escalates fast, and the code often still fails to run.

Part 2: What Is Spec-Driven Development?

Spec-Driven Development is an engineering methodology that connects human architectural intent to AI-generated syntax. Its principle is simple: the quality and stability of AI-generated code are directly proportional to the structural rigor of the context you give the model [somniosoftware, 2026].

In traditional development, the codebase is the ultimate source of truth and documentation trails behind the implementation. SDD flips that. The specification becomes the immutable source of truth, and the source code is a transient, compiled output generated by the AI [mariano-aguero, 2026]. Your job shifts from writing syntax to writing constraints. You decide the "what" and the "why," and the agent handles the "how."

How Do You Measure SDD Maturity?

Software architects including Birgitta Böckeler and Martin Fowler describe three levels of SDD maturity, based on the lifecycle of the specification [Martin Fowler, 2025]:

Level 1: Spec-First (Documentation First). You write a precise specification before implementation and give it to the agent as high-fidelity context. You and the AI may both edit the code and the spec during the build. Once the feature ships, though, the spec is often discarded or left to drift. When you need updates later, you write a new, localized spec.

Level 2: Spec-Anchored. The specification is a durable project asset. It lives in version control alongside the codebase and anchors all maintenance. When a feature evolves, you update the existing spec first. That gives the agent stable, historical context to generate new code against.

Level 3: Spec-as-Source. This is the most advanced form. The specification is the main source file. Engineers edit only the plain-language spec and never touch the underlying code directly. The AI agent compiles the entire application from the spec, which eliminates manual code drift completely.

How Does SDD Differ From Traditional Specifications?

Writing specs before code is not new, but SDD adapts the practice for AI consumption.

Traditional Waterfall PRDs are written for humans. They are exhaustive and monolithic, and they detail every feature upfront. They are too big for LLM context windows, which leads straight to context rot. SDD artifacts are modular, iterative, and built for machine parsing. They focus on immediate architectural boundaries instead of multi-year roadmaps.

Agile User Stories (e.g., "As a user, I want...") are short-lived narratives for backlog prioritization. They leave out technical constraints on purpose. When you feed one to an AI, the missing rigor forces the agent to hallucinate an architecture. SDD fills that gap by pairing user intent with explicit technical design documents.

Test-Driven Development (TDD) defines behavior before implementation, but it uses automated tests to validate human-written code. SDD uses the spec as a compilation target for an AI agent. Passing a test suite is necessary, but the agent must also show that the implementation matches the plain-language spec.

Where SDD Is Heading: Formal Verification and Executable Contracts

The most advanced SDD implementations add formal verification to plain-language specs. These are mathematical constraints that make an agent's behavior provably correct instead of merely testable [arXiv, 2026].

One research direction is λ-RLM (Recursive Language Models grounded in λ-calculus). Instead of letting an agent generate free-form code, λ-RLM restricts the model to a library of pre-verified functional combinators. The LLM handles only the smallest subproblems that fit cleanly in its context window, and deterministic, symbolic logic handles all higher-level architectural routing. That eliminates non-termination failures [arXiv, 2026].

The industry is also using formal specification languages like Dafny and Alloy to create executable checkpoint specifications. In these pipelines, a natural language spec is translated into machine-checkable constraints and inserted into the codebase as assertions. Tools like SpecCoder train coding models to treat those assertions as verifiable checkpoints. The agent has to prove that intermediate data states satisfy the spec's formal properties, not just that the final output passes a test [arXiv, 2026].

For most builders today, formal verification is an enterprise and research concern, not a practical workflow. But it shows where the methodology is going: from "specify before you build" toward "prove before you ship."

Part 3: The SDD Workflow, Idea to Shipped App

A mature SDD workflow is a structured pipeline where the order matters. Each phase separates working out what you want from building it, which protects the agent's context window from rot.

Phase 1: Elicitation and Architectural Pre-Build Planning

Before the agent generates a single line of code, define the boundaries of your system. Solo developers who jump straight to UI prompting in web-based tools most often fail here.

In this phase, you turn high-level goals into technical requirements, pick database schemas, and map third-party API dependencies. The output is an Architecture Design Document (ADD). It locks down technical decisions early and makes sure the agent understands the full dependency graph before it starts. With that map in hand, the agent is less likely to silo functions or ignore your database structure.

Phase 2: Setting Constitutional Guardrails

Every SDD project sets a few underlying laws, a project constitution. It defines your non-negotiable standards: your required tech stack, authentication patterns, compliance rules, and formatting conventions.

Store the constitution in your project directory, for example as a .cursorrules or CLAUDE.md file. Putting it in the root context means every agent session inherits your baseline constraints automatically. You no longer have to remind the agent of your structural rules in every new chat.

Phase 3: Drafting the Behavioral Contract

With the architecture and constitution in place, write the behavioral specification for your target feature. This document states exactly what the software must do.

A good spec describes what the software must achieve and which edge cases it must handle. Use behavioral Given/When/Then scenarios to make the conditions explicit. Avoid prescriptive implementation directives in this document. Leave the line-by-line syntax choices to the agent.

Phase 4: Task Breakdown and Atomic Execution

The agent reads your spec and generates a sequential implementation plan, broken into discrete tasks you can review.

To keep context clean, enforce atomic execution: give each task a fresh context window. The agent reads the spec, implements the isolated code, and runs an automated feedback loop to verify the component. Once it passes, you commit the code atomically before the agent moves on to the next task. This keeps the agent's attention budget from degrading over long sessions.

Phase 5: Verification and Convergence

In the final phase, you check that the generated code fulfills the behavioral contracts in the spec. Multi-agent systems often use dedicated verification agents to audit the implemented logic against the original markdown files.

If they find discrepancies, the system starts a localized rework loop. Once everything converges and all tests pass, you merge the artifacts. Your living documentation and your actual codebase stay in sync.

Part 4: The SDD Tooling Landscape

Tooling for SDD has evolved quickly. Options range from heavyweight enterprise governance frameworks to lightweight command-line utilities.

Platform / Framework	Core Methodology & Features	Primary Artifacts	Target Audience & Workflow Fit
GitHub Spec Kit	Enforces a strict, phase-gated pipeline (/specify, /plan, /implement). Includes a constitution module.	Markdown files under a .specify/ directory.	Greenfield projects and enterprise teams that need extreme traceability.
OpenSpec	Brownfield-first, delta-based change management. Avoids full system rewrites by specifying only what changes.	Delta specs using ADDED, MODIFIED, and REMOVED markers.	Teams maintaining legacy applications. Uses context limits to force brevity.
Zalcro	Pre-build elicitation and architectural sequencing. Turns ideas into agent-ready plans before coding begins.	Architecture Design Documents (ADD), Prompt Packs, Ticket Backlogs.	Vibe coders and agentic builders who want to avoid the 80% integration cliff.
GSD (Get Stuff Done)	Focuses on execution hygiene and preventing context rot through atomic commits.	CONTEXT.md, PLAN.md, and Git Logs.	Terminal-based developers who prioritize clean agent memory and task verification.
Intent / Brunel	Multi-agent continuous verification and living API contracts.	Verified specs synchronized actively with backend code.	Enterprise teams fighting API drift with Verifier agents.
AWS Kiro	End-to-end cloud IDE built natively around structured specs.	IDE-native spec files linked directly to infrastructure.	Developers in the AWS ecosystem who want seamless provisioning.

How to Choose Your Tooling

GitHub Spec Kit is a rigorous process harness backed by a large developer ecosystem. Its phase gates can't be skipped: an agent can't write code until the human operator has validated its technical plan. That creates a highly auditable trail, but it also creates a lot of file overhead, with multiple documentation files even for simple tasks [somniosoftware, 2026].

OpenSpec is "brownfield-first," on the view that most development means modifying existing systems. Instead of demanding an exhaustive upfront spec, it uses "Delta Specs" [glukhov.org, 2026]. It generates localized specs that describe only the changes. Once the agent implements a delta, the tool merges it into a master directory, so your full specification builds up over time.

Zalcro targets the window before the first line of code exists. As a pre-build planning layer, it translates natural language intent into machine-optimized architecture [GptZone, 2026]. For web platforms like Lovable and Bolt.new, it generates sequenced instructions you can paste in, so databases are provisioned before UI components. For terminal developers using Cursor or Claude Code, it outputs a dependency-aware ticket backlog that syncs to issue trackers and gives the agent strict architectural guardrails.

Part 5: Managing Drift and Technical Debt

The biggest threat to an AI-assisted project's longevity is specification drift. Drift is the growing gap between the software running in production and the intent the engineers originally documented [MindStudio, 2026]. As agents generate code quickly, the gap widens and technical debt (TD) piles up. Analysis of LLM-powered applications shows that repositories accumulate technical debt rapidly, and that prompt configuration and optimization issues account for a large share of the maintenance burden [arXiv, 2026].

The Three Mechanisms of Code Drift

In agentic development, code drift compounds through three main mechanisms [MindStudio, 2026]:

Prompt Drift. Your codebase collects fragmented, contradictory intent over time. Different developers use different terminology in isolated, short-lived chat sessions. The overall architectural logic disappears when the chat window closes.

Hand-Edit Drift. When a bug surfaces in production, you might skip the agent and patch the compiled code by hand. These edits bury critical business logic in the source code, and it never gets backported to your spec. That corrupts your source of truth.

Model Drift. Foundation models change quickly, and different models use different internal reasoning paths and carry different biases. If you swap or upgrade models, you introduce structural inconsistencies when modifying code generated by older versions.

Reversing Drift with Immutable Specs

Unmanaged drift makes a codebase unreadable to both humans and agents, which leads to maintenance failures and forced rewrites. SDD fixes this by making your specification an immutable reset point.

In a mature SDD environment, you are discouraged, both culturally and technically, from hand-editing source code. When a behavior needs to change, you edit the plain-language spec and tell the agent to recompile the affected code.

Enterprise platforms automate this discipline with multi-agent CI/CD pipelines. Verifier agents continuously audit the repository and cross-reference contracts and markdown specs against the application's routing logic. If an agent hallucinated a database field that isn't in the spec, or if someone changed an endpoint without updating the contract, the pipeline fails the build immediately. This two-way synchronization keeps the implementation aligned with your intent.

Conclusion

Agentic Software Engineering promises unprecedented velocity, but raw generation speed without architectural rigor degrades codebase stability. Unstructured prompting speeds up local productivity while harming repository coherence, and it drives up code churn and hidden security vulnerabilities.

Spec-Driven Development is the evolution needed to use autonomous LLMs safely and economically. It makes the specification a machine-readable, version-controlled contract, which separates your architectural intent from the agent's syntax implementation. Whether you adopt the phase-gated governance of GitHub Spec Kit, the delta specs of OpenSpec, or the pre-build architectural elicitation of Zalcro, the core point is the same: AI coding tools need rigorous orchestration. As the industry moves from celebrating individual developer speed to prioritizing team-level delivery stability, SDD gives you the foundation to build software that lasts.

FAQ

What is the AI Productivity Paradox?

Generative AI speeds up individual code creation but degrades team-level throughput and delivery stability. Rapid code generation without architectural planning overwhelms downstream review, which leads to higher rework rates.

How does Spec-Driven Development differ from traditional specifications?

Traditional specs are written for human engineers and often lack the precise technical constraints an AI needs. SDD creates modular, machine-readable artifacts that act as direct compilation targets, so the AI agent works within strict architectural guardrails.

What is context rot in AI coding?

Context rot happens when a model's context window fills up with earlier prompts, error logs, and discarded code attempts. As the attention budget runs out, the AI loses track of the original intent and starts making compounding errors. Projects often stall at the 80% cliff.

Why is token waste so high in agentic development?

The continuous agent loop drives it. The AI repeatedly guesses at fixes, reads error logs, and repopulates its context window. That cycle can consume up to 1,000 times more tokens than a standard chat interaction, inflating compute costs without producing working code.

How can developers prevent AI code drift?

Treat the specification as the immutable source of truth and avoid hand-editing source code. When something needs to change, update the plain-language spec first and let the AI agent recompile the affected code.[Anthropic, 2026] — Effective context engineering for AI agents

[Anthropic, 2026] — Effective context engineering for AI agents
[arXiv, 2026] — Speed at the Cost of Quality: Evaluating AI Code Generation Stability
[DORA.dev, 2024] — Accelerate State of DevOps Report 2024
[Falconer, 2026] — Agent loops are burning your token budget. Here's the fix.
[GitClear, 2026] — The Maintainability Gap: 2026 AI Code Quality Research
[glukhov.org, 2026] — OpenSpec Quickstart: Install, Workflow, and Common Pitfalls
[Google Cloud Blog, 2025] — Announcing the 2025 DORA Report
[GptZone, 2026] — Zalcro | AI Application
[Iternal, 2026] — Token Usage Guide 2026: How Many Tokens AI Really Uses
[Josh Owens, 2026] — Context Rot: Why Your AI Gets Dumber the Longer You Use It
[mariano-aguero, 2026] — mariano-aguero/spec-driven-development-skill
[Martin Fowler, 2025] — Understanding Spec-Driven Development: Kiro, spec-kit, and Tessl
[METR, 2026] — Task-Completion Time Horizons of Frontier AI Models
[MindStudio, 2026] — The Real Cost of AI-Generated Code Drift, and How to Stop It
[Preprints.org, 2026] — The Productivity Paradox of AI-Assisted Software Development
[somniosoftware, 2026] — Spec-Driven Development in Practice: GitHub Spec Kit, OpenSpec, and GSD Compared
[SWE-bench PRO, OpenReview, 2026] — SWE-bench Pro: Can AI Agents Solve Long-Horizon Tasks?
[YouTube, 2026] — The Real Cost of AI Agents in 2026 (Token Prices vs the Loop)

Originally published at zalcro.ai/guides/spec-driven-development

Vibe Coding Hasn't Won. The Industry Is Misreading Its Own Data.

David Odukoya — Wed, 07 Oct 2026 05:20:40 +0000

Originally published at zalcro.ai/guides/vibe-coding-hasnt-won

The most repeated statistic in software development right now is that 84% of developers use AI coding tools. We have seen it in blog posts, investor decks, conference keynotes, and LinkedIn threads declaring that software engineering has fundamentally changed. The number is real, it comes from a real survey, but that number doesn't mean what the industry thinks it means [Stack Overflow Developer Survey, 2025].

Go back to the actual survey and read the question that produced that number: "Do you currently use AI tools in your development process?" That's it. It did not ask whether AI wrote the code or whether anyone read the output. It did not ask whether the code shipped to production. It asked whether developers touched an AI tool at all, which includes something as simple as asking ChatGPT to explain an error message once a month [Stack Overflow Developer Survey, 2025].

The same survey asked a second question, one that almost nobody quotes. It defined vibe coding explicitly as "the process of generating software from LLM prompts" and asked whether that was part of respondents' professional development work. Of the 26,564 people who answered, 72% said no. Only 15.19% said yes in any degree [Stack Overflow Developer Survey, 2025].

That is not a rounding difference. That is a fivefold gap between the number the industry quotes and the number that actually describes prompt-led development. The industry has been treating a statistic about tool exposure as a statistic about how software gets written, and those are two completely different things.

It looks like data. But what does it actually measure?

The number everyone quotes answers a different question

The problem becomes obvious when you break down what "using AI tools" actually covers in practice. The Stack Overflow survey is an umbrella question. Under that umbrella sits someone using GitHub Copilot to autocomplete function names, someone asking Claude to summarize documentation, someone running an autonomous agent that writes features without human review, and someone who accepted a Copilot suggestion once and never touched it again. All four are counted the same way.

The survey's own breakdown makes this visible. Of professional developers who said they use AI tools, only 51% reported daily use. Trust moved in the opposite direction from adoption: reported trust in AI accuracy fell from 43% in 2024 to 32.77% in 2025, while distrust rose to 45.71%. Two thirds of respondents said their biggest frustration was AI solutions that were "almost right, but not quite." These are not the answers of people who have handed the keyboard to the machine [Stack Overflow Developer Survey, 2025].

This pattern extends beyond developers. Research submitted to the Wharton Business & GenAI Conference found that enterprise users routinely override or abandon AI outputs even when those outputs are accurate, not because the model failed, but because accountability structures have not kept pace with deployment [Imo et al., 2026]. Workers cannot afford to be wrong, and machines cannot be held responsible when they are. (I am a co-author of this paper.)

The agent question is where the story gets most telling. The survey defined agents explicitly as "autonomous software entities that can operate with minimal to no direct human intervention" and asked about usage. Of 31,877 respondents, 52% either did not use agents at all or stayed with simpler autocomplete tools. Another 38% had no plans to adopt them. Stack Overflow's own summary of the results was direct: "AI agents are not yet mainstream." [Stack Overflow Developer Survey, 2025]

A separate Stack Overflow pulse survey from April 2026 of about 1,100 technology professionals found that 63% rarely or never let agents run entirely on autopilot, and 60% block unapproved system changes [Stack Overflow Blog, 2026].

The 84% statistic is not a lie. It is just the answer to a different question from the one that was asked.

Statistic	Source	What was actually asked	What it measures
84% use AI tools	Stack Overflow 2025	"Do you use AI tools in your development process?"	Tool exposure
15.19% vibe code professionally	Stack Overflow 2025	"Is generating software from LLM prompts part of your work?"	Prompt-led development
4.7% of PRs are fully autonomous	LinearB 2026	Analyzed 2.7M pull requests directly	Hands-off agent output
0–20% of tasks fully delegatable	Anthropic Autonomy 2026	Asked developers directly	Actual autonomy ceiling

How the definition got stretched

Feb 2, 2025 — Karpathy coins "vibe coding" on X: throwaway weekend projects, refuse to read diffs
Mar 2025 — Simon Willison draws the line: reviewing code means it's software development, not vibe coding
Late 2025 — Collins Dictionary names it Word of the Year; definition requires neither hands-off behavior nor unreviewed output
Mar 5, 2025 — Ars Technica: is vibe coding gnarly or reckless? Maybe some of both
2026 — Google Cloud reframes it as responsible AI-assisted development
May 2026 — Karpathy distinguishes vibe coding from agentic engineering; tries to restore the boundary

The measurement problem did not happen in isolation. It followed a definitional collapse that started almost immediately after the term "vibe coding" was coined.

Andrej Karpathy introduced the phrase on February 2, 2025, in a post on X [Karpathy, 2025]. His description was specific: "There's a new kind of coding I call 'vibe coding', where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. I 'Accept All' always, I don't read the diffs anymore." He also scoped it clearly: "It's not too bad for throwaway weekend projects."

Two things in that original post matter. The defining behavior was not using an AI to help with code. It was specifically refusing to read the output. And the person who coined the term explicitly limited it to throwaway work.

Simon Willison drew the line cleanly about six weeks later [Simon Willison, 2025]: "When I talk about vibe coding I mean building software with an LLM without reviewing the code it writes. If an LLM wrote the code for you, and you then reviewed it, tested it thoroughly and made sure you could explain how it works to someone else, that's not vibe coding, it's software development."

That boundary did not hold. Collins Dictionary named "vibe coding" its Word of the Year for 2025 and defined it as "the use of artificial intelligence prompted by natural language to assist with the writing of computer code" [Collins Dictionary, 2025]. That definition requires neither hands-off behavior nor unreviewed output. By early March, Ars Technica was already framing the practice as both creative leverage and reckless abdication—treating those as normal variants under the same label [Edwards, 2025]. Google Cloud now distinguishes between "pure vibe coding" for throwaway projects and "responsible AI-assisted development," which it calls the practical application of the concept and which explicitly includes reviewing and testing code [Google Cloud Vibe, 2026].

A term that originally meant a specific abdication of responsibility now labels the responsible version of the same work. When surveys measure "vibe coding" under these expanded definitions, they are measuring something Karpathy himself said was not vibe coding. And when adoption figures for AI tools get reported alongside this diluted definition, the conflation becomes invisible.

Karpathy tried to restore the boundary in May 2026, distinguishing vibe coding from what he called agentic engineering, and warning that "you need to actually be in the loop a little bit" [Karpathy Agentic, 2026]. By then the term had already escaped.

What the telemetry actually shows

The adoption gap

If surveys struggle with definitions, telemetry should give us cleaner signal. It does, but it also has its own limits, and the picture it paints is far more constrained than the headlines suggest.

LinearB analyzed 2.7 million pull requests across 83,000 developers and 253 engineering organizations between February and May 2026 [LinearB Data, 2026]. In the top-decile organizations, the ones most aggressively adopting AI, 54% of pull requests had AI coding assistance and 45% of merged code lines were AI-written. Those are significant numbers. But fully autonomous agent-written pull requests accounted for 4.7% of the total in those elite organizations. At the top 30%, it was 1.1%. At the top 60%, it was 0.1%.

LinearB's own summary: "AI adoption is shallower than it looks."

Anthropic's research into how developers actually use Claude Code found that people make about 70% of planning decisions and Claude makes about 80% of execution decisions. But when asked how much of their work they could fully delegate, developers said 0 to 20% of tasks. The delegation is real. The autonomy is not [Anthropic Autonomy, 2026].

JetBrains surveyed more than 15,000 professional developers in 2026 and found that 90% used agents weekly. But only about 22% reported that more than 80% of their work code was fully agent-generated. The group they identify as genuine "agentic coders" is roughly 31% of respondents [JetBrains Adoption, 2026] [JetBrains Code, 2026].

The tools themselves encode the reality. GitHub's cloud agent requires an administrator to enable the policy and repository owners to opt in, has a 59-minute session limit, and its own documentation describes reviewing the diff as part of the workflow. Claude Code's GitHub Actions guidance instructs users to "review Claude's changes before merging." Cursor requires approval before executing terminal commands by default. OpenAI's own Codex documentation still states that "it remains essential for users to manually review and validate all agent-generated code before integration and execution." [GitHub Copilot Agent, 2026] [Claude Code, 2026] [Cursor Enterprise, 2026] [OpenAI Codex, 2026]

The industry narrative is running ahead of what the tools themselves are designed to do.

The downstream cost of the conflation

This isn't just a debate over definitions, the fallout is already hitting production. Anthropic ran a randomized experiment [Anthropic Skills, 2026] with 52 mostly junior engineers. The AI assistant sped up the task slightly, but not significantly. The AI group scored 50% on a post-task comprehension quiz versus 67% for the hand-coding group. Developers completed the work faster but understood it less. When someone merges a pull request they cannot fully explain, they have deferred the cognitive cost of understanding to a future incident.

This is what researchers in 2025 and 2026 started calling comprehension debt: the growing gap between the demands a codebase makes on a development team and the collective understanding the team actually has. Unlike technical debt, it is invisible. The code may look clean, tests may pass, and the team's mental model of what the system actually does may be almost entirely wrong.

The LinearB PR data [LinearB Gap, 2026] shows where this surfaces in practice. Autonomous agent pull requests sit in review queues for 17.6 hours at the 75th percentile, compared to 3.4 hours for manual work. Human reviewers actively avoid picking up AI-generated pull requests because the cognitive overhead of auditing complex, unverified logic they did not write is significantly higher than reviewing code a colleague wrote. The fundamental constraint in software engineering has shifted from generating code to reviewing and verifying it.

DORA's 2025 research captured this at scale. Across nearly 39,000 technology professionals, they found that teams were generating and deploying code faster with AI, but delivery stability did not recover alongside throughput. Their March 2026 qualitative deep dive described the pattern plainly: "Time saved writing is often re-spent auditing." [DORA, 2025] [DORA, 2026]

What is actually happening

The developers and organizations getting real value from AI in 2026 are not the ones who handed over the keyboard. They are the ones who built structure around the tool. The methodology that has emerged in response to the failures of unstructured prompting is Spec-Driven Development (SDD). Instead of describing what you want and hoping the AI produces the right thing, the developer writes a structured specification first. The specification becomes the source of truth. The AI agent translates it into code. The human's job shifts from typing syntax to defining intent precisely enough that the agent cannot misinterpret it.

The parallel shift in infrastructure is Context Engineering, sometimes called ContextOps. Research showed that simply expanding a model's context window does not improve performance. Frontier models experience severe degradation when flooded with irrelevant data. ContextOps is the practice of governing exactly what information the agent sees at each step, keeping it grounded in organizational reality rather than hallucinating patterns from its training data.

These are not vibe coding. They are the opposite of vibe coding. They are rigorous, deterministic constraints built specifically to prevent the failures that come from treating AI generation as a finished product.

Karpathy himself described the distinction in May 2026 [Karpathy Agentic, 2026]: vibe coding "is about raising the floor for everyone in terms of what they can do in software." Agentic engineering "is about preserving the quality bar of what existed before. You're not allowed to introduce vulnerabilities due to vibe coding. You're still responsible for your software just as before."

Autonomous generation hasn't taken over. The teams actually shipping code are using structured collaboration, which ironically makes the human's job harder, not easier.

Where things stand

AI assistance is close to universal among developers who answer surveys, and that part of the story is real. Agentic coding is growing fast and has crossed from novelty to material use. What has not happened is what the 84% supposedly proves. Hands-off, prompt-led production development remains a minority practice. The numbers that actually measure it tell a consistent story: 4.7% of pull requests in the most AI-adoptive organizations, 15% of Stack Overflow respondents saying prompt-generated software is part of their professional work, 0 to 20% of tasks that developers say they can fully delegate [LinearB Data, 2026] [Stack Overflow Developer Survey, 2025] [Anthropic Autonomy, 2026].

The industry drew the wrong conclusion from its own data. It reported the broadest available figure as though it measured the narrowest behavior, and skipped the denominators, review status, and post-merge outcomes that would have told a different story.

The developers navigating 2026 well are the ones who understood that while AI makes code cheap to generate, human comprehension and architectural judgment remain the scarcest resources in the software lifecycle. The tools that help them plan before they build, structure before they prompt, and verify before they ship are the ones that survive contact with production.

FAQ

Is 84% still a meaningful number?
Yes. It shows that almost every developer has touched an AI tool, but it says nothing about how they use them. If you want to know how much code is shipping without human review, you have to look at telemetry and post-merge outcomes, and those numbers are much smaller.

Does this mean vibe coding doesn't work?
It works perfectly for weekend hacks, which is exactly how Karpathy originally scoped it. The breakdown happens when the industry tries to drag a hobbyist workflow into professional software development and expects the same results.

What is the difference between AI-assisted development and vibe coding?
Simon Willison drew the clearest line. If you reviewed the code, tested it, and can explain how it works to someone else, that is software development. If you accepted the output without reading it, that is vibe coding. It ultimately comes down to whether a human can take responsibility for the merged code.

If autonomous PRs are only 4.7%, why does it feel like AI is everywhere?
Because basic AI actually is everywhere. Most developers are using autocomplete, summarizing docs, or bouncing ideas off a chat window. That low-level adoption is very real, but it's being mistaken for fully autonomous code generation.

What should developers actually be doing differently?
Write the specification before you prompt. Nail down your architecture, data models, and boundaries before an agent writes a line of code. Keep the work contained, one task, one commit, and one context window. The teams actually shipping reliable AI-generated code are putting their effort into rigorous upfront planning rather than hoping a model guesses their intent correctly.

Works Cited

[Anthropic Autonomy, 2026] — Measuring Agent Autonomy
[Anthropic Skills, 2026] — AI Assistance and Coding Skills
[Claude Code, 2026] — Claude Code GitHub Actions: Review Claude's Changes Before Merging
[Collins Dictionary, 2025] — Word of the Year 2025
[Cursor Enterprise, 2026] — Cursor Enterprise: LLM Safety and Controls
[DORA, 2025] — Accelerate State of DevOps Report 2025
[DORA, 2026] — Balancing AI Tensions, March 2026
[Edwards, 2025] — "Will the future of software development run on vibes?", Ars Technica, March 5 2025
[GitHub Copilot Agent, 2026] — About GitHub Copilot Coding Agent
[Google Cloud Vibe, 2026] — What Is Vibe Coding?
[Imo et al., 2026] — Enterprise Trust Gaps in Generative AI (unpublished preprint, co-authored by David Odukoya)
[JetBrains Adoption, 2026] — AI Coding Agent Adoption 2026
[JetBrains Code, 2026] — How Much Code Do Developers Really Let Agents Write?
[Karpathy, 2025] — Original Vibe Coding Post, February 2025
[Karpathy Agentic, 2026] — Agentic Engineering Distinction, May 2026
[LinearB Data, 2026] — AI in Software Development: What the 2026 Data Shows
[LinearB Gap, 2026] — AI Engineering Productivity Gap
[OpenAI Codex, 2026] — Introducing Codex
[Simon Willison, 2025] — What Vibe Coding Is, March 2025
[Stack Overflow Developer Survey, 2025] — Stack Overflow Developer Survey 2025: AI and Developer Tools
[Stack Overflow Blog, 2026] — Agents on a Leash: Agentic AI Remains Mostly Monitored at Work, May 2026