DEV Community: Cleber de Lima

In a scenario where Context is King, engineering the harness to provide the exact amount of context to agents perform optimally brings both precision and cost efficinecy

Cleber de Lima — Wed, 08 Jul 2026 07:06:05 +0000

Cleber de Lima

Jul 7

Your Coding Agents Are Drowning in Context: You Pay Twice, in Tokens and in Precision

#ai #programming #llm #agents

5 min read

In a scenario where Context is King, knowing how to provide the exact amount of context to agents to perform at the optimal level can bring both precision and cost reduction

Cleber de Lima — Wed, 08 Jul 2026 07:03:44 +0000

Cleber de Lima

Jul 7

Your Coding Agents Are Drowning in Context: You Pay Twice, in Tokens and in Precision

#ai #programming #llm #agents

5 min read

Your Coding Agents Are Drowning in Context: You Pay Twice, in Tokens and in Precision

Cleber de Lima — Tue, 07 Jul 2026 23:29:35 +0000

Look at what your coding agents actually pull into context. For a single task, the agent greps the repo, runs a broad vector search, and loads dozens of files and chunks that merely resemble the request into the window before it writes a line. You pay for that twice. Once in tokens, for context the model never uses and once in precision, because the few facts that matter are now buried in lookalike text, and a window crowded with noise makes the model reason worse. The reflex may be to buy a longer context window, but a bigger window only buys room for more noise.

The fix is not more retrieval or a bigger window, it is structuring what you retrieve so the agent gets exactly what the task needs, which is cheaper and sharper at the same time.

Building the AI-DLC harness at Betsson we found that replacing "search the codebase" with a knowledge layer the agents query is the best way to get that right. It can be by reverse engineer the codebase, or to build a full knowledge base fit for Agent utilization. An agent picking up a task now pulls the exact typed slice it touches, the contract it must satisfy and the decision that governs it, instead of paying to read whatever text sits near the task in embedding space.

The answer is not a better embedding model or a bigger window. It is to give the agent a structured map of the system and let it fetch only the part a task needs. That map is an ontology.

What an ontology is

An ontology is a machine-readable map of a domain: the things that exist, the typed relationships allowed between them, and the rules that constrain them.

It is the schema of a knowledge graph, defined before you extract anything. For a codebase it names entities like service, contract, data entity, and decision record, and edges like implements, depends on, and governed by. The meaning lives in the edges.

The constraint is the feature: because the agent can only traverse relationships you defined, it stops inventing them, which cuts hallucination.

Why precise context is cheaper and sharper

Start with why plain vector based RAG fails; The retrieval ranks by similarity, which is statistical, not structural, so high cosine similarity does not mean a chunk is relevant, and the agent over-fetches to compensate. The problem is not vectors, it is what you feed them. Most stacks slice documents into arbitrary token windows, so a contract gets split across two chunks and three unrelated snippets get bundled into one.

Even using more advanced techniques like Semantic Search or AST-Based code chunking, you still only have a piece of knowledge, not the relationships it carries.

Reverse the order. Let the ontology decompose your knowledge into atoms before anything is embedded: one contract, one decision, one entity per atom, self-contained and typed. Build the relationships with other atoms and present that to the agent in the format of a knowledge graph.

That pays off two ways, and they weigh the same. The token cost drops, because you stopped fetching and reading filler. And the output gets better, because Your vector proximity search now runs over a curated layer, keeping the fast recall of embeddings while gaining the precision of chunks cut along meaning giving the agent only the atoms the task needs and it reasons over signal instead of noise. From a retrieved atom it then walks typed edges to the exact dependencies, each hop governed by the schema rather than guessed from proximity.

The gain compounds. You compile the atoms and edges once, upstream, and every one you add makes the next task both cheaper and more accurate, while raw embeddings depreciate and get re-computed with each model generation.

Composable retrieval

None of this means rip out vector search. Treat retrieval as composable, and get the order right: the ontology decomposes knowledge into atoms first, you embed those atoms, and at query time you route across two components you own, vector proximity over the curated atoms for broad recall, and graph traversal over the typed edges for relational precision.

Similarity finds the entry atoms, then the query walks two or three hops to expand the exact subgraph the task needs, so the agent can both read it and write back what it learns. Keep the atoms, the schema, and the confidence scores in formats you own, so the layer is portable and governed like the rest of your platform rather than locked inside a black box.

Industry is already working on solutions to implement this type of solution, Pinecone's Nexus for example, packages this as a "knowledge engine, not a retrieval system": it compiles data into typed, cited artifacts upstream and answers a single declarative query that carries its own token budget, instead of ranking chunks at runtime and probably more solutions will emerge on this area that is a critical piece of agentic infrastructure.

The playbook to get there

Start where retrieval overspends. Find the tasks where the agent pulls the most context and uses the least of it, and the relational questions it gets wrong. Signal: tokens per task, and wrong-context answers in that area.

Decompose with the schema before you embed. Agree the entity and edge types up front, and cut your knowledge into ontology-defined atoms that become the chunks, rather than fixed-size windows. Pitfall: model-invented types and near-duplicate edges. Signal: relationship types stay closed, and chunks map to whole units of meaning.

Reverse-engineer the baseline. Capture entities, contracts, and decisions for the components in scope as versioned artifacts, not wiki prose. Signal: an agent can answer "what does this expose and depend on" from the artifacts alone.

Serve it over MCP and route composable. Broad lookups to vector proximity over the atoms, relational questions to graph traversal; return the typed slice, not the repo. Signal: context precision, the share of retrieved tokens the model actually uses.

The strategic takeaway

More retrieval is not more signal. It is a bigger bill and a more crowded window, and the answer suffers as much as the budget does. The lever is order: decompose knowledge with an ontology first, so your chunks are the exact atoms agents need, then let vector proximity and graph traversal work over that curated layer. Precise context is cheaper and sharper at once, and neither gain comes at the other's expense. RAG retrieves what looks related; an ontology defines what connects, and a composable layer lets you use both. Embeddings depreciate with every model you adopt; the ontology compounds with every atom you define.

So decide what your harness feeds its agents: more text that resembles the task, or a structured slice they can reason over. If you think a longer context window closes this, make the case in the comments. If not, ask your platform team two things about the last task an agent ran: how much of the context you paid for did the model actually use, and how much of what actually mattered was buried in the rest?

[Boost]

Cleber de Lima — Thu, 18 Dec 2025 16:13:49 +0000

Continuous Fluid Flow: How AI Is Compressing the Software Delivery Cycle

Cleber de Lima ・ Dec 18

#programming #ai #softwareengineering #softwaredevelopment

Continuous Fluid Flow: How AI Is Compressing the Software Delivery Cycle

Cleber de Lima — Thu, 18 Dec 2025 16:13:32 +0000

Your developers are 55% faster. Your pull requests take 91% longer to review. Your deployment frequency is flat or declining. Welcome to the productivity paradox of AI-enabled development.

After 15 years leading enterprise transformation programs across cloud, DevOps, and now AI, I have seen organizations repeatedly optimize the wrong constraint. AI has made coding faster, but coding was never the real bottleneck. Product decisions, quality assurance, deployment automation, and production learning loops now determine whether teams actually deliver value or simply generate code that queues up for review.

This shift demands a fundamental rethinking of how work flows through delivery systems. The two-week cadence that defined Agile for two decades was designed for human-speed development. When AI compresses what took days into hours, time-boxed iterations become containers too large for the work they hold and too slow for the feedback it needs.

The Bottleneck Has Moved

The 2025 DORA Report delivers a sobering assessment: despite 90% AI adoption among developers, delivery stability declined 7.2% in organizations using AI coding tools without adequate governance. Individual productivity metrics improved while system-level throughput stagnated or declined.

Andrew Ng puts it bluntly: the bottleneck is now deciding what to build. When prototypes that took teams months can be built in a weekend, waiting a week for user feedback becomes painful.

GitClear's analysis of 211 million lines of code reveals an 8-fold increase in duplicated code blocks between 2020 and 2024. Teams are producing more artifacts while the cognitive load on reviewers, testers, and operators accelerates beyond their capacity to absorb it.

The Continuous Flow Model

The pattern emerging from both research and practice abandons time-boxed iterations entirely. Work moves through the system as capacity allows rather than waiting for arbitrary boundaries. Each work item progresses independently from specification through generation, validation, and deployment.

The model operates through three parallel, continuous activities rather than sequential phases.

AI-intensive delivery is the primary work stream. A cross-functional assembly of product, engineering, QA, and SRE works with full dedication on well-bounded goals. AI is embedded throughout: requirement refinement, architecture options, code generation, test creation, documentation. The team operates in focused mobilization sessions where AI proposes and humans validate in real time. Work flows through quality gates as fast as it can pass them, with WIP limits preventing the system from generating more than review capacity can absorb.

Early-life support runs as a continuous responsibility. As each increment reaches production, the team monitors telemetry, triages issues, responds to user feedback, and makes rapid fixes. This happens for each deployment rather than batching support into a dedicated phase.

Learning and assetization operate as an ongoing discipline. The team continuously extracts patterns, creates reusable templates and prompts, improves automation, and shares knowledge. This is where compound advantage gets built. Without this deliberate investment, organizations accelerate artifact production while learning velocity remains unchanged.

AWS has documented similar patterns with their AI-DLC methodology. Practitioners experimenting with compressed cycles report that removing time boundaries forced them to develop critical skills around finding small slices of value and collaborating effectively. The intensity is high, but so is the learning velocity: feedback loops that took weeks now complete in hours or days.

The Playbook for Continuous Flow

Based on research from AWS, DORA, McKinsey, and on the implementation work with enterprise engineering organizations, here is a structured approach I'm using to move toward continuous flow.

Step 1: Assess Readiness Before Accelerating

What: Evaluate your team's technical practices, cultural environment, and infrastructure maturity before removing time boundaries.

Why it matters: The 2025 DORA Report's central finding is that AI does not fix teams; it amplifies what already exists. Strong teams with robust testing and fast feedback loops see gains. Struggling teams with tightly coupled systems see instability increase. 90% of organizations now have AI adoption, but 30% still do not trust AI-generated code.

How to do it: Map your current SDLC and identify where bottlenecks exist before AI acceleration. Assess platform engineering maturity: automated testing coverage, CI/CD sophistication, and observability instrumentation. Establish baseline DORA metrics. Evaluate architecture for coupling, since tightly coupled systems cannot absorb AI-generated change velocity.

Pitfall to avoid: Assuming tools alone will fix problems. AI amplifies existing dysfunction.

Metric and signal: Clear identification of your top three bottlenecks with baseline DORA metrics established.

Step 2: Scale QA and Deployment Infrastructure First

What: Implement AI-powered testing and progressive delivery infrastructure before accelerating development. QA capacity and deployment safety must expand before development velocity increases.

Why it matters: 63% of teams cite QA as their biggest delay. When AI accelerates coding without scaling testing and deployment automation, quality degrades rather than velocity improving. Elite performers in DORA's research deploy 182 times more frequently than low performers while maintaining 8 times lower change failure rates. The difference is infrastructure that enables safe experimentation.

How to do it: Pilot AI test generation that analyzes code changes to auto-generate scenarios as part of mobilisation and development, based on specifications and requirements not on ready code. Implement self-healing tests and predictive defect detection. Deploy feature flags enabling toggling features on and off, canary releases to 5-10% of users first, and automated rollback when anomalies are detected. Build observability dashboards with real-time visibility into performance and user behavior.

Pitfall to avoid: Over-reliance on AI testing without human judgment for business logic. Feature flag debt from old flags not cleaned up.

Metric and signal: Test creation time reduces 70%. Deployment frequency increases 2-5x. Change failure rate decreases despite volume increase.

Step 3: Establish AI Code Review Governance

What: Create specialized review processes for AI-generated code and deploy automated quality gates.

Why it matters: Code review has become the last-mile bottleneck. Reviewers take 26% longer for AI-heavy pull requests because they must check for hallucinated packages, pattern misuse, and duplicated code. Without governance, organizations accept risk they do not understand.

How to do it: Create AI-specific review checklists checking for hallucinated packages, business logic verification, and security vulnerabilities. Implement PR tagging requiring AI assistance percentage notation, triggering additional review for PRs exceeding 30% AI content. Deploy automated quality gates catching duplication, complexity, and maintainability issues before human review.

Pitfall to avoid: Treating automated review as replacement for human review. You need both.

Metric and signal: Review time stabilizes despite volume increase. Percentage of issues caught in automated gates exceeds 60%.

Step 4: Form Multidisciplinary Teams with AI as Collaborator

What: Assemble cross-functional teams where product, engineering, QA, and SRE work alongside AI agents in focused, uninterrupted synchronous sessions.

Why it matters: AI-native development paradoxically requires more synchronous human collaboration, not less. The traditional pattern of writing a ticket, waiting for grooming, waiting for planning, waiting for development, waiting for review stretches decisions across weeks with constant context loss. Mobilization compresses that same decision density into hours of focused collaboration. When the full team is present with AI, questions get answered immediately, decisions happen in seconds rather than days.

How to do it: Assemble volatile teams that form around well-bounded goals. Include product ownership for intent validation, engineering for technical judgment, QA for quality perspective, and SRE for operational awareness. Integrate AI agents as active collaborators embedded in every phase. Protect mobilization time ruthlessly from interruption. Structure sessions in two modes: Mob Elaboration where the team co-creates specifications with AI, and Mob Construction where AI generates while humans validate in real time.

Pitfall to avoid: Treating mobilization sessions as optional meetings rather than protected deep work. Forming teams without all necessary disciplines.

Metric and signal: Decision latency decreases from days to minutes during sessions. Output per mobilization session exceeds output from equivalent distributed async time.

Step 5: Pilot Continuous Flow with Learning Systems

What: Experiment with removing time-boxed iterations on contained work. Implement WIP limits and quality gates. Establish systems where learnings compound into organizational assets.

Why it matters: When AI enables idea-to-prototype cycles measured in hours, two-week sprints become containers too large for the work they hold. Continuous flow allows work to move through the system as fast as capacity permits. But AI accelerates delivery only if outputs compound. Repeated one-off code creates technical debt at AI speed. Organizations that build reusable prompt libraries and standardized patterns achieve higher productivity with each delivery.

How to do it: Break work into small, well-specified items with clear acceptance criteria. Implement WIP limits based on review and validation capacity. Establish quality gates that work must pass. Maintain continuous production monitoring with focused support as each increment deploys. Dedicate ongoing time to documenting learnings and improving automation as a parallel activity rather than a batched phase. Build reusable AI assets tailored to your context. Consider asset creation as part of the definition of done, not allowing tasks to be completed without the automation, prompt fine-tuning, and context curation.

Pitfall to avoid: Removing boundaries without implementing WIP limits. Neglecting learning time because it is not scheduled.

Metric and signal: Learning cycles complete 2-3x faster. Cycle time decreases as flow improves. Reuse ratio across projects increases over time.

What to Start, Stop, Continue

For Executives

Start: Treating AI adoption as operating model transformation, not tool deployment. Allocating budget for progressive delivery infrastructure, AI testing platforms, and team training. Measuring learning velocity alongside delivery velocity.

Stop: Declaring victory based on license adoption without delivery outcome linkage. Removing time boundaries without building prerequisite infrastructure. Ignoring downstream bottlenecks in QA, review, and deployment.

Continue: Investing in platform engineering as the foundation AI amplifies. Demanding evidence that AI delivers value, not just activity.

For Engineers

Start: Treating AI-generated code as untrusted input requiring validation. Building context packs and reusable prompts for your domain. Participating in mobilization sessions where AI proposes and humans validate.

Stop: Accepting AI suggestions without reviewing the code. Treating every AI interaction as an isolated transaction. Ignoring downstream effects of accelerated code generation on reviewers and testers.

Continue: Applying rigorous review standards to all code regardless of origin. Building expertise in context engineering and AI orchestration. Sharing successful patterns with the broader organization.

Strategic Takeaway

Continuous flow is not about working faster. It is about learning faster.

Time-boxed models treated learning as an outcome of shipping: deploy, measure, adjust. Continuous flow treats learning as a parallel activity: ship, support, extract patterns, compound assets, repeat. This shift from output velocity to learning velocity separates organizations building sustainable advantage from those generating code at AI speed while accumulating technical and organizational debt.

The prerequisite investment is substantial. Progressive delivery infrastructure, AI-powered testing at scale, observability-driven development, and AI code review governance are not optional enhancements. They are the foundation that makes continuous flow possible without collapse.

The 2025 DORA Report's finding is the essential insight: AI does not fix teams; it amplifies what already exists. Strong foundations plus AI acceleration equals compound advantage. Weak foundations plus AI acceleration equals compound failure.

The organizations winning in this new era will not be the ones generating the most code. They will be the ones with the tightest learning loops and the most effective knowledge compounding. Speed without learning is just motion.

If this challenges your current delivery model, that is the point. Share your perspective on continuous flow. Challenge the framework if you see gaps. The best operating models emerge from rigorous debate among practitioners who have tried these patterns in production.

[Boost]

Cleber de Lima — Thu, 18 Dec 2025 10:59:27 +0000

Building Software in the Age of AI: The Mindset Shift and the Playbook That Actually Works

Cleber de Lima ・ Nov 12

#ai #softwareengineering #productivity #development

User Stories were made for Humans, Specs are made for AI

Cleber de Lima — Thu, 18 Dec 2025 10:51:37 +0000

Cleber de Lima

Nov 20 '25

From User Stories to Machine-Ready Specs: Why Your Requirements Process is Breaking Down in the Age of AI

#ai #softwareengineering #productivity #development

5 min read

Is this the End of Agile as we know it ??

Cleber de Lima — Thu, 18 Dec 2025 10:38:56 +0000

Cleber de Lima

Dec 18 '25

The End of Agile: When the Assumptions Beneath Your Methodology Collapse

#programming #ai #agile #softwaredevelopment

12 min read

The End of Agile: When the Assumptions Beneath Your Methodology Collapse

Cleber de Lima — Thu, 18 Dec 2025 10:29:04 +0000

Every methodology is a response to constraints. When the constraints change, the methodology must change with them. Agile was a brilliant response to the constraints of its era, however that era has ended.

To understand why Agile cannot survive AI, it is essential to understand why Agile exists at all. Not the rituals and ceremonies that accumulated around it, but the fundamental assumptions about software development that made those rituals sensible.

After more than 15 years leading enterprise transformations, I have watched many organizations adopt methodologies, many of them without full understanding why those methodologies work and why they should adopt them. They copy the practices that seems to be working well without grasping the principles, transforming the methodology into a serie of ceremonies and steps, without recognizing the constraints those ceremonies were designed to address. This works until the constraints change. Then the methodology becomes a cage rather than a scaffold.

We are at that moment now where AI has not merely accelerated development. It has invalidated the assumptions on which two decades of methodology were built on.

Why Agile Was Brilliant

The Agile Manifesto emerged in 2001 when seventeen developers gathered to articulate what they had learned from successful projects. They were not theorists inventing abstractions. They were practitioners who had discovered what actually worked.

What made Agile brilliant was its precise fit to the constraints of its moment.

Software requirements in 2001 were genuinely unknowable upfront. The internet was young. Businesses were discovering what software could do for them. Users were learning what they wanted. Nobody could fully specify requirements at project start because nobody knew what the right product looked like until they saw working software. Agile embraced this uncertainty. Welcoming changing requirements was not naive optimism but acknowledgment that learning would happen throughout the project.

The cost of change had dropped dramatically. Modern programming languages, IDEs, and version control made code modification routine rather than heroic. Agile leveraged this by treating course correction as acceptable. Teams did not need everything right upfront because adjusting direction was cheap.

Communication technology had transformed what was possible. Email and instant messaging meant teams could stay synchronized without elaborate documentation. Direct human conversation became the highest-bandwidth channel available. Agile designed around this: daily standups, pair programming, co-located teams. These practices leveraged the new reality that talking was faster than writing documents.

Most importantly, human creativity was the scarce resource. Writing good software required talented people making countless decisions about architecture, algorithms, and implementation. These decisions could not be automated. Organizations could only create conditions where smart people did their best work.

Agile optimized for human performance

Trust over control, sustainable pace over death marches, self-organizing teams over command hierarchies. This recognized that human cognition was both the engine and the constraint of software development.

The methodology spread because it worked. Teams that adopted Agile delivered better software faster than teams that did not. This was not ideology. It was competitive advantage born from accurate understanding of actual constraints.

The Hidden Assumptions

Beneath Agile's practices lay assumptions so obvious in 2001 that nobody needed to state them explicitly.

Humans would do the work. Every principle assumed human developers writing code, human testers finding bugs, human architects making design decisions. Motivation mattered because humans needed motivation. Sustainable pace mattered because humans burned out. Face-to-face conversation was optimal because humans communicating with humans was the information bottleneck.

Two weeks was fast. When the manifesto suggested delivering working software frequently, from a couple of weeks to a couple of months, with preference to the shorter timescale, two weeks represented ambitious speed. Human teams genuinely needed that long to produce meaningful increments.

Working software was difficult to produce. Making software work at all was the hard part. Working software therefore served as the primary measure of progress because it indicated genuine achievement.

Requirements would be interpreted by humans. User stories could be vague because smart developers would fill gaps, ask clarifying questions, and apply judgment. The feedback loop from customer to team to code to customer had humans at every step, interpreting and translating at each handoff.

These assumptions were so deeply embedded that they became invisible. Nobody questioned whether humans would do the work because what else would do it? Nobody questioned whether two weeks was fast because how could it be faster?

AI has made these invisible assumptions visible by breaking them.

How AI Changes Everything

The cost of code generation has collapsed. What once took developers days now takes AI minutes. Agile assumed producing code was expensive because human effort was expensive, so the methodology optimized for producing less code more carefully. AI inverts this completely. Generation is now cheap. The constraint is no longer writing code but specifying what code to write and validating that the result is correct.

Two weeks is no longer fast. AI-enabled development produces working prototypes in hours. A two-week sprint is not rapid iteration in this context. It is an artificial delay that queues work behind an arbitrary time boundary. When concept-to-working-code happens in an afternoon, waiting twelve more days for a sprint boundary serves no purpose except ceremony compliance.

Human communication is no longer the highest-bandwidth channel. Agile optimized for face-to-face conversation because that was the fastest way for humans to transfer information to other humans. AI agents consume entire codebases instantly. They maintain perfect context across thousands of files. They never forget previous decisions. The bandwidth of human conversation, once the solution, has become the constraint.

Working software is no longer the meaningful measure of progress. When AI generates working software from specifications in minutes, the software itself is not the accomplishment. The specification that accurately captures intent is the accomplishment. The validation that confirms correctness is the accomplishment. Code has become an intermediate artifact, not the end product.

Requirements can no longer be vague. Agile tolerated imprecise user stories because humans interpreted them, asked questions, and applied judgment. AI interprets specifications literally. Vague input produces wrong output. The precision that human developers provided implicitly must now be provided explicitly in specifications. This is not a minor adjustment. It inverts the Agile preference for working software over comprehensive documentation. When AI generates the software, the documentation is what matters.

The feedback loop no longer requires humans at every step. Specifications can generate code without human interpretation. Validation can be automated against specifications. Testing can be generated alongside implementation. Humans remain essential for judgment calls, but they are no longer needed for translation at every stage.

Human stamina is no longer the pacing constraint. AI agents do not burn out. They do not need sustainable pace. The humans who remain in AI-augmented teams need protection, but they are doing different work. They sustain a pace of specification, review, and decision-making, not a pace of code production.

Self-organization means something different when the team includes AI. The tacit knowledge that self-organizing human teams surfaced through collaboration must now be made explicit so AI can act on it. Architecture decisions, coding standards, domain models, interface contracts: everything humans once held in their heads must be written down as context for AI consumption.

Each of these changes alone would require methodology adjustment. Together, they invalidate the foundation on which Agile was built.

Why Agile Is No Longer the Best Approach

Agile optimized for constraints that no longer bind. It treated human effort as the scarce resource when AI has made generation cheap. It treated two weeks as fast when hours is now possible. It treated working software as the achievement when specifications and validation are now the hard parts. It treated human communication as the solution when human communication bandwidth is now the bottleneck.

Organizations running Agile in AI-enabled environments experience characteristic dysfunctions. Code review times expand dramatically because AI generates code faster than humans can evaluate it. Sprint boundaries create artificial delays as completed work waits for ceremonies. Estimation becomes meaningless when AI execution time bears no relation to human effort estimates. Standups consume time sharing information that automated systems could surface instantly.

These are not implementation failures. They are methodology mismatch. The practices that optimized for human-paced development actively impede AI-accelerated development.

The deeper problem is that Agile's fundamental orientation is wrong for the new constraints. Agile asks: how do we help humans produce software effectively? The question for AI-native development is different: how do we specify intent precisely, validate output rigorously, and apply human judgment where it matters most?

Methodologies Built for the New Constraints

Several frameworks have emerged that address AI-native constraints directly.

AWS AI-Driven Development Lifecycle replaces sprints with Bolts: intense cycles measured in hours or days rather than weeks. It introduces Mob Elaboration and Mob Construction, sessions where cross-functional teams co-create specifications with AI in real time. Human judgment concentrates at approval gates rather than distributing across every implementation decision.

Spec-Driven Development treats specifications as executable contracts. Code generates from specifications and regenerates when specifications change. The specification becomes the source of truth; code becomes derived output. This directly addresses the precision requirement that AI imposes.

Continuous Flow models abandon time-boxed iterations entirely and represent perhaps the most natural fit for AI-augmented development. In continuous flow, work moves through the system as capacity allows rather than waiting for sprint boundaries. Each work item progresses independently from specification through generation, validation, and deployment. There is no batching into two-week containers because AI does not naturally operate in two-week increments.

The mechanics of continuous flow address AI-specific constraints directly. Work-in-progress limits prevent the system from generating more code than review capacity can absorb. This is essential because AI can produce artifacts far faster than humans can evaluate them. Without WIP limits, review queues explode and the verification bottleneck chokes delivery.

Prioritization in continuous flow happens continuously rather than at planning ceremonies. When market conditions shift or critical issues emerge, priorities change immediately. Work does not wait for the next sprint planning session to be reprioritized. This matches the responsiveness that AI-accelerated execution makes possible.

Quality gates replace phase boundaries. Instead of requirements phase, development phase, testing phase, continuous flow implements gates that work must pass: specification review, generation validation, security scanning, integration testing, deployment approval. Work flows through gates as fast as it can pass them. Nothing waits for artificial time boundaries.

Measurement shifts from velocity to flow metrics. Cycle time tracks how long work takes from start to done. Throughput tracks how many items complete per period. These metrics reveal actual delivery performance without the distortions that story points and velocity introduce.

Continuous flow also enables genuine single-piece flow where each feature or fix moves through the entire system independently. AI agents can work on implementation while humans work on the next specification. The system operates as a pipeline with multiple items in flight at different stages rather than as a batch processor that completes everything in a sprint before starting the next batch.

This is not abandoning Agile's wisdom. It is applying Agile's deepest insight.

These methodologies share common elements: specifications before generation, human judgment at decision points rather than every step, continuous flow rather than artificial batching, validation capacity as a first-class constraint.

The Playbook for Transition

Moving from Agile to AI-native methodology requires deliberate transformation rather than gradual drift.

Step 1: Identify the Actual Bottleneck

Before changing methodology, it is critical to understand where the delivery system actually constrains. Measuring cycle time decomposed into its components reveals the truth: how long does work spend in specification, in development, in review, in testing, in deployment? Tracking waiting time separately from working time exposes hidden delays. Most organizations assume development is the bottleneck when it has already shifted to review and validation. Methodology should address actual constraints, not assumed ones.

Step 2: Pilot Continuous Flow on Contained Work

Selecting a team and work stream for pilot allows experimentation without organization-wide risk. Internal tools, greenfield features, or well-defined technical improvements work well. The pilot removes sprint boundaries for this work, implements WIP limits based on review capacity, and establishes quality gates that work must pass. Work flows through the system as fast as gates allow. Running the pilot long enough to generate meaningful data, typically eight to twelve weeks, provides evidence for broader adoption.

Step 3: Build Specification Discipline

AI-native development requires precise specifications. Training product and engineering teams to write specifications that define inputs, outputs, constraints, acceptance criteria, and edge cases explicitly is foundational work. Even here AI can help, by validating and transforming ambiguous requirements in very detailed specifications. Establishing specification review as a quality gate ensures precision before generation begins. Iterating on specification formats until AI reliably produces correct output from them takes time but pays compound returns. This is the most difficult cultural change because it inverts the Agile preference for minimal documentation.

Step 4: Embrace Focused Mobilization

AI-native development paradoxically requires more synchronous human collaboration, not less. The shift is from distributed async work punctuated by ceremonies to intense focused sessions where cross-functional teams mobilize together. Mob Elaboration sessions bring product, engineering, and design together to co-create specifications with AI in real time. Mob Construction sessions concentrate human judgment at critical decision points while AI handles generation. These sessions work because they eliminate context switching. When the full team is present with AI, questions get answered immediately, decisions happen in seconds rather than days, and feedback loops compress from sprint cycles to minutes. The traditional pattern of writing a ticket, waiting for grooming, waiting for sprint planning, waiting for development, waiting for review stretches decisions across weeks with constant context loss. Mobilization compresses that same decision density into hours of focused collaboration. Teams report that four hours of synchronized mob work with AI produces more validated output than weeks of distributed async work. The key is intensity and focus: short bursts of complete attention rather than fragmented attention spread across days. This requires protecting mobilization time from interruption and treating these sessions as the primary unit of work rather than as meetings that interrupt real work.

Step 5: Restructure Around Verification

Recognizing that generation is no longer the bottleneck changes how teams organize. Building review capacity as infrastructure rather than afterthought becomes essential. Implementing AI-assisted review tools to catch routine issues frees human reviewers to focus on judgment calls. Breaking large AI-generated changes into reviewable units prevents review queue overflow. Establishing clear criteria for what requires human review versus automated validation creates sustainable flow. Treating review capacity as a planning constraint alongside development capacity ensures the system does not generate more than it can verify.

Step 6: Implement Flow Metrics

Replacing velocity and story points with flow metrics provides visibility into actual performance. Tracking cycle time from work start to deployment breaking the time between each phase, reveals where delays occur. Tracking throughput as items completed per period shows delivery rate. Tracking WIP ensures limits are respected. Tracking quality metrics ensures speed does not degrade correctness. Making these metrics visible to the organization enables data-driven improvement. Using them to identify constraints and drive improvement creates a learning system.

Step 7: Retire Agile Ceremonies Explicitly

As continuous flow takes hold, formally deprecating ceremonies that no longer serve purpose prevents overhead accumulation. Sprint planning becomes continuous prioritization. Daily standups become async status updates or focused problem-solving sessions when genuine blockers arise. Retrospectives shift from process improvement to specification and context improvement, to ensure the knowledge and learn become assets to compound for continuous improvement. Allowing old and new processes to run in parallel indefinitely creates waste. Explicit retirement prevents ceremony accumulation.

Step 8: Scale What Works

Once the pilot demonstrates improvement, expanding to additional teams spreads the benefit. Adapting based on pilot learnings addresses context-specific needs. Different teams may need different WIP limits or gate configurations. Maintaining measurement discipline as scaling proceeds catches degradation early. Watching for problems that indicate the model is not transferring correctly allows course correction. Expecting the full transition to take twelve to eighteen months for a large organization sets realistic timelines.

What to Start, Stop, Continue

For Executives

Start examining the assumptions beneath current methodology. Pilot continuous flow on contained work streams. Measure cycle time and throughput rather than velocity and sprint completion. Build review and validation capacity as strategic infrastructure.

Stop treating Agile as permanent infrastructure. Stop measuring success by ceremony compliance. Stop assuming two-week iterations are inherently correct. Stop expecting AI to accelerate delivery without methodology change.

Continue demanding evidence that methodology produces results. Continue investing in engineering excellence. Continue building capacity to adapt as constraints keep evolving. Most important, continue to ensure value is delivered iteratively and constantly - the soul of agile.

For Engineers

Start learning specification-driven practices. Build skills in AI orchestration and output validation. Understand why current practices exist, not just what they are. Experiment with flow-based work on individual tasks.

Stop defending ceremonies without examining their assumptions. Stop treating methodology as religion. Stop accepting practices because they are familiar rather than because they are effective.

Continue focusing on outcomes over process. Continue maintaining quality standards regardless of what generates the code. Continue adapting as the craft evolves.

Strategic Takeaway

Agile was a response to the constraints of 2001: unknowable requirements, expensive code changes, human effort as the scarce resource. The methodology succeeded because it accurately addressed those constraints. Two decades of adoption reflect two decades of competitive advantage for teams that understood what Agile was actually optimizing for.

AI has changed the constraints. Generation is cheap. Verification is expensive. Specifications must be precise. Human judgment, not human effort, is the scarce resource. Flow matches AI capability better than time-boxed iteration. The methodology that served brilliantly for twenty years now optimizes for constraints that no longer bind while ignoring constraints that now dominate.

The transition to AI-native methodology is not optional for organizations that intend to remain competitive. When competitors move from idea to production in hours while others wait for sprint boundaries, methodology becomes market disadvantage. The playbook is clear: understand actual bottlenecks, pilot continuous flow, build specification discipline, embrace focused mobilization with tight feedback loops, restructure around verification, measure flow rather than velocity, and retire ceremonies that no longer serve purpose.

This is not abandoning Agile's wisdom. It is applying Agile's deepest insight: methodology must match constraints. The constraints have changed. The methodology must change with them.

If this challenges conventional thinking about software delivery, that is the intention, The frameworks that will define the next era are being built now by practitioners grappling with real constraints in real organizations. This is the moment to shape that future rather than inherit it.

The Velocity Trap: Why Your AI Productivity Gains Are an Illusion

Cleber de Lima — Mon, 01 Dec 2025 13:27:47 +0000

You Adopted AI, Your developers are shipping more code than ever. Your pull requests have doubled. Your task completion metrics are through the roof. And your software delivery is getting worse.

This is the Velocity Trap. It is the defining paradox of AI-assisted development in 2025, and most engineering organizations are falling into it without realizing it.

If you have read one of my previous articles you know I'm an advocate of using AI to transform the product development life-cycle, but this transformation is no free of trade-offs and risks that needs to be understood and mitigated, In this article I'll discuss about 2 of these risks that are a result of an overreliance in AI for software development:

Quality and reliability of AI generated code - The 2025 DORA report reveals that AI adoption correlates with a 7.2% reduction in delivery stability despite improvements in individual output metrics. GitClear's analysis of 211 million lines of code shows an 8x increase in duplicated code blocks and a 39.9% decrease in refactoring activity. Teams are producing more artifacts while building less value.
The Impact on the formation and evolution of entry level engineers - A Stanford study shows a 13% relative decline in employment for early-career engineers in AI-exposed roles while senior positions remain stable. Companies have quietly stopped hiring juniors, preferring to invest in senior engineers who can leverage AI effectively.

The Productivity Paradox Nobody Is Measuring

A rigorous study on AI coding productivity came from METR in mid-2025. Researchers ran a randomized controlled trial with experienced developers across 246 real-world coding tasks. The finding was stunning: developers using AI tools were 19% slower than the control group. The critical detail is that those same developers believed they were faster.

This disconnect explains everything. The brain's reward system responds to AI-generated suggestions the same way it responds to solving problems ourselves. We feel productive. We press tab and accept the suggestion. The dopamine hits. But the code that accumulates is not the code we would have written, and increasingly, it is not code we fully understand.

The downstream effects appear in metrics most teams are not tracking. Code review times have increased 91% for teams heavily adopting AI. Pull request sizes have grown 154%. The bottleneck has shifted from writing code to reviewing it, and reviewers are drowning in AI-generated artifacts they cannot absorb at the pace they arrive.

The Amplifier Effect

The uncomfortable truth: AI does not fix teams. It amplifies whatever dynamics already exist.

The same DORA report suggests how we should think about AI in software delivery. Rather than treating AI as a universal accelerator, the researchers demonstrate that AI functions as a systemic amplifier. It magnifies existing organizational patterns, whether those patterns are strengths or dysfunctions.

High-performing teams with strong foundations use AI to eliminate repetitive work, freeing capacity for architecture and complex problem-solving. Their delivery velocity increases while quality remains stable.

Teams constrained by process or technical debt experience the opposite. AI generates code faster than their review processes can absorb. Their CI/CD pipelines buckle under increased load. Defect rates and duplicate code climbs. The organization responds by adding more process, which slows delivery further, which increases pressure to use AI more aggressively. The spiral accelerates.

51% of engineering leaders now view AI's impact as negative despite 90% of developers reporting positive sentiment.

The Junior Developer Crisis

Besides the quality and overall slowness effects, perhaps the most consequential long-term risk is what AI overreliance is doing to the engineering talent pipeline. The tasks AI handles best, boilerplate code, simple bug fixes, test generation, are precisely the tasks that have historically served as the learning ground for junior developers. When AI handles this work, the traditional on-ramp to engineering expertise disappears.

This creates a recursive problem. Current AI models were trained on code written by humans with deep understanding. We are now teaching developers to learn from AI output rather than first principles. Each iteration becomes more removed from fundamental understanding.

The phenomenon known as "vibe coding" captures this perfectly. Developers can produce complex applications without understanding how they work. When the AI-generated code breaks, these developers are helpless. They can generate more code. They cannot debug what they have.

By 2030, we face a potential crisis where organizations cannot find engineers capable of making architectural decisions or debugging systems at a fundamental level, since they are not learning now how to do it.

The Playbook for Mitigating Over-Reliance Risks

The path forward is not to abandon AI. Adoption is irreversible and AI is here to stay. The path forward is to move from unmanaged velocity to governed agility, from measuring output to measuring outcomes.

Step 1. Treat AI Code as Untrusted Input

Create explicit policies requiring engineer-in-the-loop validation for all AI-generated contributions. Not just any reviewer, but qualified engineers who understand system context and can evaluate architectural fit, security implications, and business logic. Tag AI-generated code for tracking so you can measure its performance in production separately. Make "never merge code you have not read and understood" a non-negotiable norm.

Pitfall: Creating policies without enforcement. Build validation requirements into your CI/CD pipeline.

Metric: Defect rates in AI-generated versus human-written code.

Step 2. Implement Multi-Layer Quality Gates

Deploy automated quality analysis at multiple points: pre-commit analysis catching basic issues, pull request enhancement augmenting human review with AI analysis, and continuous monitoring tracking quality trends over time. Research shows teams using AI-assisted code review see 81% quality improvement compared to 55% without.

Use AI to enhance and automate your testing strategy. AI can build unit, contract and integration tests very efficiently, not only increaseing the coverage, but by understand patterns, behaviours, analysing telemetry and cover the most critical paths with edge cases a human would find it difficult to do it. Capitalize on that to reduce the bugs and performance issues before moving AI generated code into production

Pitfall: Treating AI review as replacement for human review. You need both.

Metric: Quality improvement from baseline, security vulnerability escape rate. Defect rates in AI-generated versus human-written code.

Step 3. Build Context and Reusability as Infrastructure

This step separates organizations that get exponential returns from those that accelerate technical debt. The difference is whether AI outputs compound into organizational assets or evaporate as disposable code.

Treat context as persistent, versioned infrastructure. Create project-specific context files encoding your architecture, standards, and constraints. Structure context packs with description, inputs, outputs, constraints, and examples. Right-size to interfaces and contracts, not full repositories. Too little context and AI hallucinates. Too much and noise drowns signal.

Design for compound engineering: workflows where multiple AI capabilities stack. Generative models produce code, predictive models select test coverage, optimization models tune performance. Each capability feeds the next. Instruct AI to produce modular components with clear interfaces designed for reuse. Use meta-prompts like "Optimize for reusability and clarity across modules." Build feedback loops where production results and code reviews update your context packs and guardrails automatically.

Pitfall: Treating every AI interaction as an isolated transaction. That creates technical debt at AI speed.

Metric: Reuse ratio across projects, reduction in duplicate patterns, first-pass acceptance rate over time.

Step 4. Preserve Skills and Rebuild the Talent Pipeline

The junior developer crisis is not a future problem. It is happening now.

Schedule pair programming sessions and debugging exercises using manual tools. Skills atrophy without use. When AI fails on edge cases, engineers who have lost fundamental skills cannot recover.

Do not eliminate entry-level roles. Reimagine them. The tasks AI handles, boilerplate, simple bugs, test generation, can become learning exercises rather than production shortcuts. Use AI as an explanation engine: have juniors ask AI to explain generated code line by line, then explain it back to a senior. Understanding why code works matters more than producing it. Train critical evaluation through AI code review: juniors review AI-generated code specifically to find flaws, security issues, and architectural violations. This develops judgment faster than writing code from scratch because they see more patterns in less time. Have juniors write context files and prompts for the team. Crafting effective AI instructions requires deep domain understanding, forcing them to learn the system architecture and business logic to communicate it precisely.

The role of the engineer is shifting from code author to AI orchestrator. Invest in the skills this era demands: prompt engineering, context management, critical evaluation of AI output, architectural thinking, security review, and debugging AI-produced code.

Pitfall: Assuming engineers will figure it out. Self-directed learning fails for the majority who need structure and deliberate practice.

Metric: Skill retention assessments, debugging proficiency without AI, junior-to-senior progression rates.

Step 5. Measure Outcomes, Not Output

Establish baseline metrics before expanding AI adoption: deployment frequency, change failure rate, mean time to recovery. Add quality metrics: bug escape rate, code duplication percentage, refactoring rate. Track developer experience: satisfaction scores, trust in AI outputs, time spent debugging AI code versus manual code. If a metric does not change behavior when it moves, stop tracking it.

Pitfall: Focusing only on velocity metrics while ignoring quality degradation.

Metric: Clear correlation between AI usage and delivery outcomes.

What to Start, Stop, Continue

For Executives

Start: Treating AI adoption as operating model transformation, not tool rollout. Measuring delivery outcomes alongside productivity metrics. Investing in training with dedicated budget. Linking AI goals to performance reviews. Reimagining junior roles instead of eliminating them.

Stop: Declaring victory based on adoption percentages. Measuring success by lines of code. Cutting junior headcount without reimagining entry-level development. Expecting instant ROI without accounting for the proficiency curve. Assuming skills will develop without structured investment.

Continue: Investing in engineering excellence as the foundation AI amplifies. Demanding evidence that AI delivers value, not just activity. Building the mentorship infrastructure that develops future senior talent.

For Engineers

Start: Treating AI-generated code as untrusted input. Learning prompt engineering and context management as core professional skills. Building and maintaining context files for your projects. Practicing regularly without AI to maintain fundamental capabilities. Mentoring juniors on how to evaluate and improve AI output.

Stop: Accepting tab completions without reading the code. Expecting AI to understand implicit context. Relying on AI for problems you cannot solve yourself. Letting AI become a substitute for developing architectural thinking.

Continue: Applying the same rigor to AI-generated code as any code review. Building expertise that makes you effective with or without AI. Investing in the fundamentals that let you know when AI is wrong.

Strategic Takeaway

The organizations winning in AI-assisted development are not the ones generating the most code. They are the ones generating the most value with the highest integrity.

The winning strategy is not AI First. It is Engineering First, AI Accelerated. Build the foundations that AI amplifies: robust governance, quality engineering practices, context discipline, and continuous skill development. Measure what matters, outcomes rather than output, and be willing to slow down when the metrics indicate degradation.

AI is a kinetic weapon that amplifies both competence and dysfunction. Deploy it into a mature engineering organization with strong discipline, and it multiplies capability. Deploy it into a struggling organization without foundations, and it accelerates collapse.

The junior developer crisis adds urgency. If we do not invest in skill development and rebuild the talent pipeline now, the question "who will be senior in 2035?" will have no good answer. The engineers who thrive in this new era will be those who can work effectively with or without AI, who understand systems deeply enough to know when AI output is wrong, and who develop the judgment that only comes from deliberate practice and struggle.

The question is not whether to use AI. That decision is already made. The question is whether you will use it in ways that build sustainable advantage, develop your people, and create long-term organizational capability, or in ways that create short-term illusions while undermining everything that makes great engineering possible.

If this challenges your current AI strategy, that is the point. Share your perspective. Challenge the framework. The best operating models emerge from rigorous debate, not comfortable consensus.

Testing Reinvented: Why Test Coverage Is the Wrong Metric

Cleber de Lima — Mon, 24 Nov 2025 17:07:36 +0000

When testing consumes considerable amount of your development cycle, AI changes everything. But most organizations are optimizing for the wrong goal.

I have guided engineering organizations through every major technology evolution over the past two decades, including the migration from manual QA to automated suites, waterfall phases to continuous testing in DevOps pipelines.

The AI transformation is different. It requires reconceiving what testing means and who does it.

Traditional testing treated quality as verification. Write code, write tests (or vice versa when using TDD), run tests, fix bugs. AI makes that sequence obsolete. When AI generates comprehensive test suites in hours, analyzes production telemetry to identify untested paths, and predicts failures before they happen, the bottleneck shifts from test creation to test strategy. The constraint is no longer how many tests we write but which tests matter.

Why Traditional Testing Metrics Fail in the AI Era

Test coverage is a vanity metric. It measures what percentage of code has been executed, not whether the right behaviors are validated or critical risks are addressed. Teams hit coverage targets while shipping production failures because they measured execution, not effectiveness.

The problem deepens with AI-generated code. When AI produces hundreds of lines in seconds, writing tests to cover those lines becomes trivial. But those tests validate syntax without interrogating logic, check happy paths without exploring edge cases, and verify implementation details instead of business intent. Coverage numbers climb while quality stagnates.

Traditional testing operates reactively. Developers write code, then tests, then discover problems, then fix them. When AI generates prototypes in hours, this sequential approach creates bottlenecks. Organizations accelerate development but maintain waterfall testing phases, optimizing artifact velocity while leaving the fundamental constraint untouched.

Tools like ContentSquare and Google Analytics consistently reveal that users interact with applications in ways developers never anticipated. They access features in unexpected sequences, use mobile devices for desktop-designed workflows, and encounter edge cases that seemed improbable during development. The gap between tested scenarios and real-world usage represents systematic risk that traditional testing never addresses.

The required shift: from measuring activity to measuring outcomes. Not how many tests exist but which risks are mitigated. Not coverage but effectiveness.

The New Paradigm: From Reactive Testing to Predictive Quality Engineering

AI transforms testing from verification into a continuous intelligence system operating as an integrated loop: AI generates tests from specifications before code exists, predicts failure modes based on code patterns and historical data, validates behavior continuously as code evolves, learns from production telemetry to identify gaps, and feeds insights back to improve specifications and future strategies.

Testing moves upstream. Instead of writing tests after code, AI generates comprehensive test suites from requirements before implementation begins. These tests become executable contracts that guide development rather than trailing indicators.

Testing becomes predictive. AI analyzes code patterns, architectural decisions, and historical failure data to identify high-risk areas before testing begins.

Testing operates continuously. Rather than batch testing at phase gates, AI validates every change in real time. Developers receive immediate feedback on what broke, why it matters, and which downstream systems are affected. Cycle time from commit to validated build drops from hours to minutes.

Testing learns. Production telemetry and user behavior analytics feed back to test generation. When users encounter edge cases or user behaviour tools reveals workflow abandonment or feature usage patterns diverging from design assumptions, these insights become test cases. The test suite evolves based on actual usage patterns.

Quality engineering emerges as a distinct discipline. QA professionals shift from manually executing test scripts to designing test strategies, evaluating AI-generated test effectiveness, establishing quality signals and thresholds, governing risk-based testing approaches, and orchestrating feedback loops between testing, development, and production operations.

The Five-Step Playbook for AI-Native Testing

Step 1. Generate Tests from Specifications, Not Code

What: Use AI to create comprehensive test suites directly from requirements, design documents, and API contracts before implementation begins.

Why it matters: Test-Driven Development has always been the gold standard but rarely practiced because writing tests before code requires effort and discipline. AI eliminates the friction. When tests exist before implementation, they guide development rather than trailing it.

How to do it: Provide AI with structured specifications including inputs, expected outputs, constraints, edge cases, and failure scenarios. Use tools like GitHub Copilot or Cursor to generate test scaffolding. Create property-based tests that validate behavior across input ranges rather than specific examples. Generate contract tests validating API agreements between services. Establish test templates encoding your organization's quality standards so AI-generated tests inherit these patterns automatically. Implement specification reviews before development to ensure tests validate the right behaviors.

Pitfall to avoid: Generating tests from existing code rather than specifications. That validates what was built, not what should have been built. The test suite becomes a mirror of implementation rather than a contract for correctness. If requirements are ambiguous, AI generates ambiguous tests. Invest in specification clarity before test generation.

Metric and signal: Percentage of tests generated before implementation. Time from specification to executable test suite. Defect detection rate in AI-generated versus human-written tests. Developer feedback on whether tests clarified requirements before coding.

Step 2. Implement Risk-Based Testing with AI Prediction

What: Use AI to analyze code complexity, change patterns, historical failures, and architectural dependencies to predict where defects are most likely and concentrate testing effort accordingly.

Why it matters: Uniform test coverage wastes resources. Not all code carries equal risk. A critical payment processing module demands more rigorous validation than a cosmetic UI adjustment. AI makes risk assessment systematic and data-driven.

How to do it: Implement AI-powered risk scoring evaluating cyclomatic complexity, recent change frequency, historical defect density, number of dependencies, security sensitivity, and production incident correlation. Use tools like Microsoft's AI-assisted testing framework or build custom risk models using your organization's historical data. Establish risk tiers with explicit testing requirements. High-risk changes require comprehensive test coverage, security scanning, performance validation, and manual review. Medium-risk changes get automated functional testing and architectural review. Low-risk changes receive smoke tests and automated validation only. Create feedback loops where production incidents automatically elevate risk scores for affected modules.

Pitfall to avoid: Treating AI risk scores as deterministic rather than probabilistic. AI predictions guide resource allocation but do not replace engineering judgment. A low-risk score means apply appropriate rigor relative to actual risk, not skip testing. Overriding AI recommendations should be easy when context justifies it but tracked so patterns inform future models.

Metric and signal: Correlation between AI risk scores and actual production defects. Reduction in testing time while maintaining or improving defect detection. Percentage of high-severity production incidents flagged as high-risk during testing. Engineering satisfaction with risk-based testing approaches.

Step 3. Build Continuous Validation Loops

What: Integrate AI testing throughout the development workflow so every code change receives immediate validation feedback rather than waiting for batch test runs.

Why it matters: Delayed feedback creates rework. When developers discover test failures hours later during CI pipeline runs, they context-switch away from the problem. Immediate validation enables correction while cognitive context is fresh. Defects caught within minutes cost 10 times less to fix than defects discovered hours or days later.

How to do it: Implement AI-powered validation at multiple integration points. In the IDE, AI provides real-time feedback as developers write code, identifying potential issues before commit. During code review, AI analyzes changes and automatically generates relevant tests or identifies missing test coverage for critical paths. In CI pipelines, AI selects which tests to run based on code changes rather than executing the entire suite, reducing build times from hours to minutes. After deployment, AI monitors production telemetry and generates tests for observed edge cases or unexpected behaviors. Establish quality gates with clear criteria at each integration point. Create dashboards showing validation results in real time.

Pitfall to avoid: Generating too many tests that slow the development cycle. AI can produce thousands of tests easily. More tests do not equal better quality. Focus on test effectiveness, not volume. Establish thresholds for test execution time and prune low-value tests regularly. Balance thoroughness with velocity.

Metric and signal: Time from code commit to validation feedback. Percentage of defects caught before code review versus during testing versus in production. Developer productivity measured by feature delivery velocity with quality maintained. Test execution time trends to ensure pipelines remain fast as test suites grow.

Step 4. Evolve Tests with Production Learning

What: Use production telemetry, user behavior analytics, and incident data to continuously improve test strategies and generate new tests that validate real-world usage patterns.

Why it matters: Developers cannot anticipate every edge case or usage pattern. Users find scenarios that test suites miss. The gap between what developers test and what users actually do represents untested risk.

How to do it: Implement multiple data streams capturing different dimensions of production reality. Technical telemetry from APM tools and logging platforms captures error conditions, performance anomalies, resource utilization patterns, and security events. User behavior analytics from ContentSquare, Google Analytics, Mixpanel, or Amplitude reveals how users actually interact with your application: navigation paths taken versus paths assumed, feature usage frequency and adoption rates, abandonment points where users leave workflows incomplete, device and browser combinations triggering issues, rage clicks and error frustration indicators, and session replay data showing exact user experiences during failures.

Use AI to synthesize these data streams and identify critical testing gaps. A ContentSquare heatmap showing users repeatedly clicking a non-interactive element indicates missing feedback that testing never validated. Google Analytics revealing 40 percent of users access a feature on mobile despite desktop-only design exposes untested responsive behavior. Session replays capturing checkout failures on specific browser and payment method combinations generate precise test scenarios.

Automatically generate tests reproducing these real-world patterns. Connect User behaviour tools to your test management platform through APIs. Configure alerts that trigger test generation when behavior anomalies exceed thresholds. When production incidents occur, AI generates comprehensive regression tests validating both the technical fix and the user experience. Tag tests with their origin whether specification-based, code-based, telemetry-based, or analytics-based so you understand your test portfolio composition.

Pitfall to avoid: Treating every production event or user behavior as a test case. ContentSquare might show thousands of interaction patterns. Google Analytics reveals countless navigation paths. Focus on critical paths, conversion flows, security issues, data integrity problems, and user-impacting failures. Establish criteria for when production observations warrant new tests: frequency thresholds for behavior patterns, business impact of affected workflows, correlation with errors or abandonment, and security or compliance implications.

Metric and signal: Percentage of test cases derived from production data and behavior analytics versus developer assumptions. Reduction in repeat production incidents. Correlation between high-traffic user paths from analytics and test coverage for those paths. Reduction in unexpected user behavior reported by support teams.

Step 5. Redefine QA as AI Quality Supervision

What: Transform QA professionals from test script executors to AI quality engineers who design test strategies, evaluate AI effectiveness, and govern quality standards.

Why it matters: Manual testing cannot keep pace with AI-accelerated development. Organizations that invest in QA evolution see quality improve while testing costs decline.

How to do it: Train QA teams on AI testing tools, prompt engineering for test generation, risk-based testing methodologies, and metrics measuring test effectiveness rather than coverage. Redefine QA responsibilities to include designing quality strategies that AI executes, reviewing AI-generated tests for completeness and relevance, establishing quality thresholds and acceptance criteria, governing test frameworks and standards across teams, analyzing quality trends and recommending improvements, and partnering with engineering to build testability into architecture and design. Create new career paths for AI quality engineers with clear progression from test execution to quality strategy to organizational quality leadership. Provide premium tools and training to QA professionals who embrace the transition.

Pitfall to avoid: Assuming all QA professionals will adapt to AI-centric roles. Some will embrace the transition. Others prefer manual testing. Support both groups but make clear that manual testing is a declining path. Offer retraining resources and transparent communication about role evolution timelines. Gradual evolution with support enables success.

Metric and signal: QA satisfaction scores with new tools and responsibilities. Percentage of QA time spent on strategy versus execution. Quality metrics including defect escape rate, time to detection, and production incident trends. Organizational perception of QA value before and after transformation.

What to Start, Stop, Continue

For Executives

Start: Measuring test effectiveness rather than coverage. Allocating budget for AI testing platforms and QA retraining. Treating quality as a continuous intelligence system. Establishing clear career paths for QA professionals evolving to quality engineering roles.

Stop: Demanding higher coverage percentages without measuring defect detection. Cutting QA headcount because AI automates testing without investing in AI quality supervision capabilities. Treating testing as a cost center to minimize. Accepting production incidents as inevitable when AI-powered predictive testing could prevent them.

Continue: Investing in engineering excellence and quality discipline. Demanding evidence that testing strategies deliver results. Supporting experimentation with new testing approaches. Building organizational capabilities for continuous learning from production.

For Engineers

Start: Generating tests from specifications before writing code. Using AI risk scoring to prioritize testing effort. Integrating continuous validation into your development workflow. Contributing production learnings back to test strategies. Treating QA professionals as quality engineering partners.

Stop: Measuring testing success by coverage percentages. Writing tests only after code is complete. Ignoring test failures because they seem flaky. Assuming AI-generated tests are automatically correct without review. Viewing testing as someone else's responsibility.

Continue: Applying rigorous review standards to all tests whether human or AI generated. Advocating for quality at every stage of development. Sharing successful testing patterns with your organization. Demanding that architecture and design prioritize testability.

Strategic Takeaway

Testing is not becoming automated. Testing is becoming intelligent.

The organizations that understand this distinction are building sustainable competitive advantage. Automated testing executes predefined scripts faster. Intelligent testing predicts where failures will occur, generates validation strategies that match actual risk, learns continuously from production, and evolves to match how systems are actually used.

This transformation requires reconceiving what quality means in an era where code generation is cheap and validation is sophisticated. Test coverage optimizes for execution activity. Test effectiveness optimizes for risk mitigation and behavior validation. That shift changes everything about how engineering organizations approach quality.

Organizations clinging to coverage metrics and phase-gate testing will build AI-accelerated technical debt. They will generate more tests that validate less. Organizations embracing test effectiveness and continuous quality intelligence will deliver faster with fewer production failures because they are optimizing for the right outcomes.

In software delivery, quality is not just a feature. It is the foundation of everything else. Speed without quality creates fragility. Features without reliability erode trust. Testing reinvented means quality engineering elevated from cost center to strategic capability.

If this challenges your current testing approach, that is the point. The organizations winning in the AI era are the ones willing to question their assumptions and rebuild their operating models around what actually works.

Share your perspective if you are rethinking testing strategy. Challenge this framework if you see gaps.

The best operating models emerge from debate, not consensus. Engineering and product leaders need to shape this transformation together because how we ensure quality is changing faster than most organizations are adapting.

From User Stories to Machine-Ready Specs: Why Your Requirements Process is Breaking Down in the Age of AI

Cleber de Lima — Thu, 20 Nov 2025 10:15:09 +0000

User stories were built for humans. AI needs something fundamentally different.

And the gap between these two realities is creating a hidden crisis in software delivery that most organizations have not even diagnosed yet.

Having led enterprise transformations through many evolution cycles, I have seen this pattern before: new capabilities arrive, we force them into existing processes, then wonder why the promised productivity gains never materialize. With AI, this mistake is particularly costly because the technology amplifies whatever clarity or confusion you provide. Feed it ambiguity, get confusion at scale.

The User Story Problem

Traditional user stories work brilliantly for human developers who fill gaps with context, experience, and intuition. They understand that "display search results quickly" means sub-second response times. They infer security requirements, accessibility standards, and architectural constraints.

AI has none of this implicit knowledge. When you tell AI to "create a search feature that displays results quickly," it generates code with no pagination, no error handling, no security controls. The AI is doing exactly what you asked, with no ability to infer what you meant.

The industry is rapidly acknowledging that we need a new contract between humans and AI. GitHub has launched the spec kit and AWS has launched KIRO with an embbeded spec-driven workflow. Leading organizations are building their own specification-driven frameworks that transform vague intentions into executable specifications, with constraints, security and architectural guidelines. These are not external tools but custom libraries that live alongside code, versioned in the same repositories, evolving through parallel refinement by AI, product managers, architects, and engineers.

Requirements as Living Code

The most successful teams I work and collaboarate with have stopped treating requirements as documents that live in Jira or Confluence. They are building specification libraries that are as integral to their repositories as the code itself. These specifications are versioned, tested, and continuously refined through the same pull request process that governs code changes.

Imagine a repository where alongside your /src directory, you have /specs containing machine-readable requirements that AI consumes directly. Product managers submit pull requests to refine business logic. Architects review and enhance technical constraints. Engineers add implementation notes. AI continuously validates consistency and completeness. This is not future vision. This is happening now.

The New Language of AI Collaboration

The transformation from user stories to AI-ready specifications requires structured intent expression that your team owns and evolves:

Traditional: "As a customer, I want to filter search results by price range so I can find products within my budget."

Team-Owned Specification: A versioned spec file in your repo containing input schema, validation rules, edge case behaviors, performance constraints, integration contracts, and concrete examples. This specification becomes the single source of truth that AI references, tests validate against, and documentation generates from.

Teams are creating domain-specific specification languages tailored to their business. An e-commerce company might have product search patterns. A fintech might have transaction processing templates. These become organizational assets, refined over months, encoding institutional knowledge in forms both humans and AI can consume.

AI-Augmented Parallel Refinement

The breakthrough is not just AI helping with requirements but the parallel refinement process where multiple intelligences collaborate simultaneously. During a typical refinement cycle, AI agents scan for ambiguities and missing edge cases, product managers validate business intent and outcomes, architects ensure system coherence and patterns, and engineers verify implementation feasibility.

Organizations report discovering requirements they would have missed until production. The AI spots patterns across your entire specification library that humans would never connect. Architects ensure consistency with system-wide constraints. Product managers maintain strategic alignment. Engineers ground everything in implementation reality.

Specification Libraries as Organizational Assets

Forward-thinking teams are building reusable specification libraries that become more valuable than code libraries. These contain patterns for common features (authentication, search, checkout), constraints for compliance and security, design systems, architectural and technology definitions, integration contracts between services, and validation rules for data quality.

When starting a new feature, teams do not begin with blank user stories. They compose from proven specification patterns, customize for specific needs, and let AI generate implementation from validated specs. The specification library grows smarter with each iteration, encoding lessons from production issues, successful patterns from high-performing features, and refined constraints from security reviews. This literally reduces the development time from weeks to hours.

Product Management Transformed

Product managers may evolve from document writers to specification architects working directly in repositories in parallel with the engineers, designers and architects. They are learning to express intent in structured formats, collaborate through pull requests, and think in patterns rather than features.

Their value lies in three capabilities AI cannot replicate:

Strategic intent curation: Ensuring every specification traces to business outcomes while maintaining consistency across hundreds of interdependent features.

Parallel validation orchestration: Coordinating refinement between AI, architects, and engineers while maintaining velocity.

Pattern recognition: Identifying which specifications should become reusable patterns and which remain feature-specific.

The Playbook for Specification-Driven Development

Step 1. Build Your Specification Library

Create a /specs directory in your repositories. Define your team's specification schema covering inputs, outputs, constraints, and examples. Start with one domain and expand gradually. Version specifications alongside code. Measure reduction in clarification cycles.

Step 2. Implement Parallel Refinement

Establish workflows where specifications are refined simultaneously by AI (completeness checking), product (intent validation), architecture (system coherence), and engineering (feasibility review). Use pull requests for specification changes. Create automated validation pipelines.

Step 3. Develop Reusable Patterns

Extract common specifications into templates. Build domain-specific languages for your business. Create specification inheritance hierarchies. Document pattern usage and evolution. Track pattern reuse rates.

Step 4. Version Requirements with Code

Stop treating requirements as external artifacts. Include specifications in code reviews. Tag specification versions with releases. Build traceability from specs to implementation. Generate documentation from specifications.

Step 5. Measure Specification Maturity

Define maturity stages for specifications. Track progression through stages. Correlate maturity with delivery success. Identify patterns that accelerate maturation. Build feedback loops from production to specifications.

What to Start, Stop, Continue

For Executives

Start: Building specification libraries as organizational assets. Treating requirements as code requiring version control. Investing in parallel refinement workflows. Measuring specification maturity as leading indicator.

Stop: Keeping requirements in separate tools from code. Using story points for AI-assisted development. Treating specifications as one-time artifacts.

Continue: Emphasizing outcomes over features. Investing in product management evolution. Maintaining focus on customer value.

For Engineers

Start: Contributing to specification libraries. Reviewing specs in pull requests. Building domain-specific specification languages. Creating specification validation tools.

Stop: Accepting external requirements documents. Working from ambiguous stories. Treating specifications as PM-only responsibility.

Continue: Collaborating on intent architecture. Maintaining excellence standards. Building reusable patterns.

Strategic Takeaway

Organizations succeeding with AI have recognized that specifications are not documentation but executable artifacts that deserve the same rigor as code. They live in repositories, evolve through pull requests, and are refined in parallel by human and artificial intelligence.

The winners will be teams that build specification libraries encoding their unique business logic, accessible to both humans and AI. These become competitive moats: the better your specifications, the faster AI can help you build, the more you learn, the stronger your specifications become.

The future belongs to organizations that treat requirements as living code, refined in parallel by diverse intelligence, versioned with the same discipline as production systems. This is not about better documentation. It is about building a new development paradigm where human intent and machine execution merge seamlessly.

We are all learning this new language together and there is a lot of space to evolve and mature the practice.

Feel free to share what works for you and discuss your challenges

The best patterns will emerge from practitioners in the trenches, not from consultants in ivory towers.