DEV Community: Remo H. Jansen

Your Platform Engineering Investment Is Not Paying Off. Here's Why.

Remo H. Jansen — Wed, 21 Oct 2026 12:50:56 +0000

Building an internal platform is only half the job. The real return comes when developers trust it, want to use it, and help make it better.

Platform engineering has become a strategic investment for many engineering organisations. Teams are building internal developer platforms, standardising CI/CD pipelines, creating self-service infrastructure, and providing tools designed to make developers more productive.

The promise is compelling: reduce cognitive load, eliminate duplicated effort, improve consistency, strengthen security, and let product teams focus on delivering business value rather than managing infrastructure.

Organisations invest significant time, money, and engineering talent to make this happen.

And yet, something often gets lost along the way.

The platform gets delivered. The migration happens. The documentation is published. The new workflow becomes the official standard.

But developers are still frustrated.

They encounter confusing errors, unreliable workflows, missing capabilities, and problems that require them to ask another team for help. They learn which workarounds are necessary, which parts of the platform they can trust, and which problems are easier to avoid altogether.

Eventually, they comply with the new way of working. But compliance is not enthusiasm, and adoption is not necessarily success.

The organisation has invested in a platform, but it has not fully realised the return on that investment.

Why does this happen, and what can we do about it?

1. The last 10% matters more than you think

Platform teams are often very good at delivering the big things.

They build the infrastructure, establish common patterns, automate the happy path, integrate the necessary services, and provide a consistent interface for developers.

These are difficult and valuable achievements.

The problem is what happens next.

The platform works, but not always reliably. The common workflow is straightforward, but the unusual case is painful. An error occurs, but the message doesn't explain what went wrong or how to fix it. A pipeline fails, and determining whether the problem is in the application, the configuration, or the platform itself becomes an investigation.

None of these problems necessarily looks significant in isolation.

Collectively, however, they define the developer experience.

Consider two internal platforms that provide exactly the same capabilities.

With the first, a developer can complete a task in minutes when everything works, but diagnosing a failure takes hours and requires support from the platform team.

With the second, the task takes the same amount of time when everything works, but failures are clearly explained, diagnostics are available, and the developer can resolve most problems independently.

Technically, both platforms may be equivalent. In practice, they deliver very different levels of value.

This is the last-mile problem in platform engineering.

We invest heavily in capabilities, but sometimes underestimate the effort required to make those capabilities dependable, intuitive, and pleasant to use.

The last 10% might be the least glamorous part of the roadmap. It is also where much of the difference between a usable platform and a genuinely valuable one is created.

2. Developers don't fall in love with roadmaps. They fall in love with experiences.

Imagine that your organisation announces a major investment in a new developer platform.

There is an ambitious roadmap, a dedicated team, a polished launch presentation, and a clear strategic rationale.

Developers are encouraged to migrate their applications and embrace the new tooling.

Initially, there is curiosity. People try the platform, explore its capabilities, and perhaps even get excited about what it could eventually become.

Then they start using it for real work.

They encounter friction. Some workflows are slower than expected. Documentation doesn't cover their situation. Failures are difficult to diagnose. Requests for improvements disappear into a queue.

Over time, the initial excitement fades.

Developers stop exploring new capabilities. They stop reporting minor bugs because the process feels disproportionate to the problem. They stop suggesting improvements because they don't expect much to happen. They learn the minimum required to get their jobs done.

The platform becomes something they have to use rather than something they want to use.

This distinction matters.

When developers genuinely value a tool, they become its advocates. They recommend it to colleagues, experiment with new features, report problems, suggest improvements, and invest time in helping it evolve.

When they merely tolerate it, they do the minimum required.

This creates a vicious cycle:

The platform has usability problems.
Developers become reluctant to engage with it.
The platform team receives less useful feedback and fewer contributions.
Problems remain unresolved or are discovered later than they otherwise would be.
The developer experience deteriorates, reinforcing the original reluctance.

The result is a platform that may look successful on a dashboard but is not reaching its full potential.

You cannot mandate genuine enthusiasm. You have to earn it through the experience you deliver.

3. The hidden cost of making developers dependent on the platform team

A successful internal platform should increase developer autonomy.

Developers should be able to provision resources, deploy applications, understand failures, and solve common problems without having to coordinate with a central team for every step.

Yet many platform implementations inadvertently create new dependencies.

A pipeline fails, so the developer raises a ticket.

A configuration behaves unexpectedly, so they contact the platform team.

A feature is missing, so they submit a request and wait.

A bug needs fixing, but the relevant code is owned by another team, and the only available path is through its backlog.

The platform may have automated many manual tasks, but the organisation has replaced one kind of dependency with another.

This is particularly costly because platform teams are usually responsible for shared capabilities used by many product teams. Every unnecessary support interaction consumes capacity that could otherwise go towards improving the platform itself.

There is also an opportunity cost for the consuming teams. Developers spend time waiting, investigating workarounds, explaining issues, and coordinating with people who understand the platform better than they do.

A platform that requires constant human intervention to operate has not fully delivered on the promise of self-service.

Of course, some interactions with platform specialists are necessary. Complex infrastructure, security-sensitive changes, and architectural decisions often need expert involvement.

The goal isn't to eliminate the platform team. It is to eliminate unnecessary dependencies on it.

A good platform enables developers to solve the problems they are equipped to solve, while making it easy to involve specialists when their expertise is genuinely needed.

4. Why can't developers help fix the platform?

Here is a scenario that should be familiar to many engineering teams.

A developer encounters a failure in a shared deployment pipeline. They suspect a defect in the pipeline itself rather than their application.

What happens next?

Perhaps they open a ticket in an IT service management system. They provide logs, explain what they were doing, and wait for someone to investigate.

Another team encounters the same problem. They open another ticket, provide similar information, and wait too.

Eventually, a platform engineer investigates both reports, identifies the underlying defect, implements a fix, and releases an update.

This process might work. But it is unnecessarily expensive if the developers encountering the problem could have collaborated directly with the maintainers.

Now imagine a different workflow.

The developer navigates to the shared platform's source repository and asks an AI coding agent to investigate the failure.

The agent examines the available logs and configuration, helps determine whether the problem originates in the application or the shared platform, and searches existing issues for similar reports.

If an issue already exists, the developer can contribute evidence and confirm that another application is affected. If no issue exists, the agent helps prepare a new one with useful diagnostic information.

Other teams can discover the issue, contribute their findings, and discuss a solution in the same place.

Perhaps a developer from one of those teams identifies the root cause and proposes a fix through a pull request. The maintainers review it, run the necessary tests, and merge it.

The platform team still owns the platform. It still defines standards, reviews changes, and ensures quality.

But it is no longer the only team capable of improving it.

This is the difference between a support model and a contribution model.

In the first, consumers report problems and maintainers solve them.

In the second, consumers and maintainers collaborate to solve problems together.

That is the foundation of an inner-source culture: applying the collaborative practices of open source within an organisation.

5. AI makes this opportunity much more compelling

The contribution model described above isn't entirely new. Inner-source practices have existed for years.

What is changing is the cost of participation.

Historically, contributing to a shared platform could require considerable effort. A developer needed to understand the repository structure, locate the relevant code, reproduce the problem, navigate contribution guidelines, and prepare a useful issue or pull request.

For someone unfamiliar with the platform, that initial investment could be substantial.

AI coding agents can reduce this barrier.

With appropriate repository access and tooling, an agent can help developers:

Understand unfamiliar platform code.
Diagnose failures using logs, configuration, and source code.
Determine whether an issue is likely to originate in the application or the shared platform.
Search existing issues and documentation for known problems.
Produce well-structured bug reports with relevant diagnostic evidence.
Identify potential fixes and prepare draft pull requests.
Write or improve tests to reproduce a defect.

These capabilities do not guarantee correct diagnoses or safe changes. Agents can be wrong, miss context, or propose fixes that introduce regressions. Human review, automated testing, access controls, and appropriate security boundaries remain essential.

But the potential is significant.

AI can make it easier for developers to cross the boundary between consuming a platform and contributing to it.

This is particularly valuable in large organisations, where the people most affected by a platform defect may be the people best positioned to reproduce it, understand its impact, or even propose a solution.

The platform team doesn't have to become less important. Its expertise can be applied where it creates the most value: architectural direction, quality, reliability, difficult problems, and enabling contributions from across the organisation.

6. Your tooling choices can either enable or undermine collaboration

Culture matters, but culture doesn't exist independently of the systems people use every day.

If source code lives in one place, issues in another, documentation in a third, and technical discussions in a fourth, contributing to a shared platform becomes harder than it needs to be.

Developers must navigate multiple systems, maintain separate accounts or permissions, repeat information, and reconstruct context that is scattered across different locations.

The organisation may have all the necessary tools. Yet the overall workflow remains fragmented.

This is especially problematic when the official process encourages developers to report a problem through a ticketing system but offers no straightforward path to investigate the underlying code or contribute a fix.

A more collaborative approach is to bring technical work closer to the source.

For code-centric platform engineering, a repository-centred workflow can provide a natural home for:

Source code and version history.
Issues, defects, and feature requests.
Technical documentation and contribution guidelines.
Pull requests, reviews, and design discussions.
Automated testing and continuous integration.

GitHub is one example of a platform that supports this model. Other tools can support similar workflows. The specific product is less important than the principle.

The place where a problem is reported should also be a place where people can understand it, discuss it, and contribute to fixing it.

There may still be good reasons to retain separate systems for service management, compliance, incident response, or formal approvals. The objective isn't to force every organisational process into a single tool. It's to avoid unnecessary separation between the technical artefacts and the people trying to improve them.

7. Platform engineering needs a product mindset

Perhaps the most fundamental change is to how we define success.

Many engineering initiatives are organised around delivery milestones: build the platform, release the capability, migrate the applications, and declare the project complete.

But internal platforms aren't finished when they launch.

They are products with users, competitors, changing requirements, operational costs, and an ongoing need for improvement.

Those users happen to be colleagues rather than external customers. That doesn't make their experience any less important.

A product mindset means treating feedback as a continuous input, prioritising usability alongside capability, and measuring whether the platform actually helps developers achieve their goals.

It also means accepting that the best feature to build next may not be a new capability.

It might be fixing an unreliable workflow. Improving an error message. Making a common failure self-diagnosable. Reducing the time required to onboard an application. Removing a manual approval that no longer adds value.

These improvements may be less impressive in a launch announcement, but they can have a substantial impact across hundreds or thousands of developers.

The platform team should therefore ask not just, "What should we build next?" but also, "What prevents developers from getting value from what we have already built?"

That second question is often where the most immediate returns can be found.

8. Measure value, not just activity

If we want platform investments to deliver better returns, we need to measure outcomes rather than relying exclusively on delivery metrics.

Counting onboarded applications, available features, pipeline executions, or registered users can tell us something about reach and activity. They don't necessarily tell us whether developers are better off.

A more useful measurement framework might include:

Developer experience

How satisfied are developers with the platform?
How often do they encounter friction in common workflows?
Do they trust the platform for critical work?

Reliability and productivity

How frequently do platform-related failures occur?
How long does it take to diagnose and resolve them?
How much developer time is spent waiting for support or working around limitations?

Autonomy and self-service

What proportion of common problems can developers resolve independently?
How often do they need human intervention?
How long does it take to onboard an application or complete a routine task?

Contribution and continuous improvement

How easy is it to report a defect with enough information to act on it?
Can consuming teams contribute changes?
How long does it take to review and release a valid contribution?
Are developers actively helping improve the platform?

These measures should be interpreted together rather than turned into isolated targets. For example, a high number of pull requests isn't necessarily evidence of success if those contributions are low quality or reflect a platform full of defects.

The aim is to understand whether the platform is reducing friction, enabling autonomy, and delivering value to the teams that depend on it.

9. What should organisations do differently?

The answer isn't to abandon platform engineering or to assume that every problem can be solved by changing tools.

The opportunity is to improve how platforms are built, operated, and evolved.

Here are five practical starting points.

1. Prioritise the last mile.

Reserve meaningful capacity for reliability, usability, diagnostics, documentation, and the removal of recurring friction. Don't allow new feature development to consume the entire roadmap.

2. Make common failures self-diagnosable.

Invest in actionable error messages, structured logs, troubleshooting guidance, and diagnostic tooling. Developers should be able to understand what failed and what to do next.

3. Make contribution a first-class workflow.

Document how to report issues, propose changes, run tests, and contribute fixes. Make it possible for consuming teams to participate without needing to become platform experts first.

4. Use AI to lower the cost of participation.

Where appropriate, provide agents with access to the relevant source code, documentation, issues, and diagnostic tools. Let them help with investigation and contribution, while retaining human oversight and appropriate safeguards.

5. Measure whether developers actually benefit.

Gather feedback, track reliability and resolution times, and examine the friction developers encounter in real workflows. Use that evidence to determine what to improve next.

None of these requires a massive transformation programme to get started. A team can begin with one frequently failing workflow, one repository, and a clear commitment to making problems easier to diagnose and fix.

The important thing is to close the feedback loop.

Conclusion: The return on platform engineering is earned after delivery

Platform engineering offers enormous potential. Shared infrastructure, standardised workflows, self-service capabilities, and reusable tooling can make engineering organisations more productive, consistent, and secure.

But building these capabilities is only part of the job.

The return on that investment depends on what happens when developers use the platform every day: whether it works reliably, whether problems are easy to solve, whether feedback leads to improvements, and whether the people who depend on it can help make it better.

A platform that developers are required to use can achieve broad adoption without earning their trust.

A platform that developers trust can achieve something much more valuable: voluntary engagement, advocacy, and continuous improvement.

That is why the last 10% matters. It is why self-service must mean more than automating the happy path. And it is why collaboration should be built into the platform's operating model rather than treated as an administrative process around it.

The next evolution of platform engineering should not simply be about building more platforms or adding more capabilities.

It should be about making existing platforms better to use, easier to understand, and easier to improve.

The real measure of a platform isn't how much engineering went into building it. It's how much friction it removes for everyone who depends on it — and how effectively those people can help it evolve.

That is how platform investments become lasting engineering value.

From Never Getting Selected to a 1-in-3 Success Rate: Did AI Help Me Write Better CFPs, or Is AI Selecting Them?

Remo H. Jansen — Mon, 19 Oct 2026 15:25:47 +0000

For years, I was involved in developer advocacy.

I organised meetups, spoke at many of them, and was invited to conferences by people in my network. At one point, I was even a Microsoft MVP, partly as a result of all that developer advocacy work.

I had experience. I had technical topics to talk about. I had a network of people who knew me and my work.

But there was one thing I could never seem to get right: getting selected through a conference Call for Papers (CFP).

I tried. A lot.

I submitted proposals to conference after conference, and the result was almost always the same: rejection.

It was frustrating. I could be invited to speak at a conference by someone who knew my work, yet struggle to get selected when submitting a proposal through the official process.

Eventually, I started wondering whether the problem was the content of my talks or the way I presented them.

This year, I tried something different

Instead of starting with a conference CFP and trying to write a proposal from scratch, I started with something I was already comfortable doing: writing a blog post.

I would choose a topic I wanted to speak about and write an article about it. Then I would give an AI agent two things:

My blog post.
The website of the conference I wanted to submit a proposal to.

I would ask the agent to use my article as the foundation for a conference proposal, taking the event's audience, themes, and submission guidelines into account.

The idea was simple: I would focus on the technical content and the ideas I wanted to share, while AI would help me turn them into a proposal tailored to the conference.

And then something unexpected happened.

The results changed dramatically

For years, I had submitted proposals without getting selected.

With this new approach, I experienced a one-in-three acceptance rate.

That's a striking difference from my previous experience.

I can't prove that AI alone caused the improvement. I may have chosen different topics, targeted different conferences, or simply submitted proposals that were a better fit. But the change was significant enough to make me wonder what was really happening.

Had I finally found a better way to sell my ideas?

Or was something else going on?

The question I can't stop asking

There are two possible explanations that interest me.

Possibility 1: AI is better at selling my ideas than I am

Perhaps I was never particularly good at writing conference proposals.

That doesn't mean I wasn't good at presenting, explaining technical concepts, or engaging an audience. Those are different skills.

A CFP proposal is a piece of persuasive writing. It has to convince a reviewer that a particular talk deserves a limited slot in a conference programme. It needs to communicate the value of the session, identify its audience, explain what attendees will learn, and distinguish itself from other submissions.

Maybe I was good at the technical side of developer advocacy but not as good at selling the value of my talks on paper.

If so, AI is helping me communicate the value of my own ideas more effectively.

There's nothing inherently wrong with that. We already use editors, colleagues, mentors, and writing tools to improve how we communicate.

But it does raise an uncomfortable question: how many good talks have been rejected over the years because their authors weren't particularly good at writing CFP proposals?

How much technical talent have conferences missed because the people evaluating submissions couldn't see the value behind an imperfect abstract?

Possibility 2: Are the reviewers using AI too?

Here's where things get more interesting.

What if the improvement isn't just about AI helping me write better proposals? What if AI is also involved in evaluating them?

Some conference organisers may use LLMs to help review, summarise, categorise, or shortlist CFP submissions. Others may rely entirely on human reviewers, and many may use a mixture of approaches.

I don't know whether the conferences that accepted my proposals used AI in their selection processes.

But if they did, it raises a fascinating possibility.

Perhaps my AI-generated proposals are better at communicating with the people evaluating them. Or perhaps they are better at matching the language, structure, and criteria that an AI-assisted review process is designed to identify.

If an LLM helps me frame my proposal and another LLM helps evaluate it, have I improved my communication with the conference organisers, or have I learned to produce something that another AI system is more likely to recognise as a strong submission?

Those are not necessarily the same thing.

A proposal can be clear, well structured, and closely aligned with a conference's stated themes without the underlying talk necessarily being more original or valuable.

Conversely, an excellent talk might be overlooked because its proposal doesn't communicate its strengths effectively.

Are we getting better at communicating ideas, or just better at optimising the text that gets those ideas selected?

I don't have an answer. But I think it's a question worth asking.

Maybe the real problem was never my technical expertise

There's another possibility: the two explanations aren't mutually exclusive.

AI may be helping me write better proposals while also making my submissions more aligned with the selection criteria, regardless of whether a human or an AI-assisted process is involved.

Giving an agent both my article and the conference website provides useful context. Rather than asking it to produce a generic abstract, I'm asking it to connect my ideas to a specific audience and event.

That could lead to clearer titles, more focused descriptions, more explicit learning outcomes, and a stronger explanation of why the session belongs at that conference.

These are useful improvements whether the reviewer is a person, an LLM, or a combination of both.

The important distinction is that a better proposal doesn't automatically mean a better talk. It means the value of the proposed talk may be easier to understand.

And perhaps that was the missing piece all along.

How I'd approach this experiment now

If you're a developer advocate, engineer, technical leader, or aspiring conference speaker, here's the workflow I'd recommend experimenting with:

Write the article first. Develop the technical ideas in your own words before worrying about selling them.
Give AI the conference context. Include the conference website, CFP criteria, target audience, and submission requirements.
Ask for a tailored proposal. Have the agent explain the talk's value to that specific audience, rather than simply summarising your article.
Review everything yourself. Make sure the proposal is technically accurate, reflects what you'll actually present, and still sounds like you.
Track the outcomes. Record submissions, acceptances, rejections, topics, and the approach used to create each proposal.

If I wanted to investigate the bigger question, I'd go further. I'd compare proposals written entirely by me with AI-assisted versions of the same ideas, ideally across comparable conferences and with enough submissions to draw more meaningful conclusions.

I'd also want to know how much AI is actually being used in CFP evaluation. Without that information, it's impossible to conclude that AI-assisted submissions are succeeding because they are being evaluated by AI.

My one-in-three result is an observation, not a controlled experiment.

But it's an observation that has changed how I approach conference submissions.

The question I'd like to leave you with

For years, I struggled to get my proposals accepted despite having experience speaking at meetups, organising communities, and contributing to developer advocacy.

Then I changed how I created my proposals, introduced AI into the process, and started seeing a very different result.

Maybe AI is simply helping me communicate my expertise better.

Maybe it is helping me optimise for a selection process that increasingly involves AI.

Maybe it's a bit of both.

I don't know yet.

But I find the question much more interesting than the simple conclusion that AI can write a better abstract.

If AI can help us write proposals that get selected, and AI is also helping evaluate proposals, what exactly are we optimising for: better talks, better communication, or better compatibility between AI systems?

I'd genuinely like to hear from conference organisers, CFP reviewers, and other speakers.

Are you using LLMs to write or refine your submissions? Are you using them to review submissions? And have you noticed a difference?

Because I suspect this is a conversation the developer community needs to have.

Halo is the new Doom and most software will soon be effectively open source (by force or by choice)

Remo H. Jansen — Sun, 18 Oct 2026 21:50:00 +0000

Seven months ago I published “Source code is now a common good, and SaaS is mostly dead.” I argued that AI had collapsed the economic cost of clean-room reimplementation, that pure software expression was losing its scarcity, and that the SaaS model built on high-margin code would face an existential squeeze—except where real data moats or network effects remained.

I was wrong about the timeline.

It didn’t take five to ten years.

This month the premonition became visible reality.

Halo everywhere — including 100+ player co-op on PC

In the space of a few days we watched Halo: Combat Evolved appear on platforms it was never meant for, and get radically expanded where it already existed:

Halo 64 (Jack Tulli) — a playable multiplayer (and emerging campaign) port running on real Nintendo 64 hardware. Built on the halo-ce-universal decompilation, with a custom renderer, four-player split-screen, the original 13 maps, bots, classic modes, and even a Firefight mode the original Xbox version never had. Claude assisted heavily in the coding.
Halo: Dreamcast Evolved — extended gameplay of the Pillar of Autumn and other levels running on actual Dreamcast hardware. The CPU is more than three times slower and the system has a quarter of the RAM of the original Xbox, so performance is still rough, but the core systems are already ported and being optimized in public.
Native PC (and beyond) with massively expanded player counts — the same decompilation work that enabled the console ports also produced native Windows, Linux, and Android builds (OpenCE / halo-ce-universal and related projects). These support system-link / online multiplayer for up to 128 players. Campaign co-op limits have been raised as well; 32-player campaign co-op is already being demonstrated, with work continuing toward the same high player counts. Browser versions of the original Xbox game have also appeared with 128-player rooms and split-screen co-op.

These are not asset flips or simple ports. They rest on prior decompilation work that reached near byte-matching fidelity. What changed is the speed of the remaining work. A single developer (or small team) plus modern agents can now finish the last mile—new renderer, platform integration, expanded netcode, extra modes—in days or weeks instead of years. The result is a franchise that once defined the Xbox suddenly running on N64, Dreamcast, modern PCs, phones, and browsers, with player counts the original hardware could never support.

Halo is becoming the new Doom: the game that runs on everything, because the barriers that used to stop it have collapsed.

Demon's Souls Remake: from ~1 FPS to 60+ (and nearly 120) in roughly a month

PS5 emulation provided the other high-visibility stress test.

In early September, Demon's Souls Remake on KyTyPS5 and its forks was crawling at 1–2 FPS even on top-tier hardware after finally reaching in-game states. Within days it jumped into the 10–18 FPS range. By mid-September temporary 30 FPS markers appeared. By mid-October, custom builds were delivering over 60 FPS at 1440p with original PS5 graphics settings on an RTX 5070 + Ryzen 7 9800X3D, and approaching 120 FPS with DLSS Frame Generation. The full game had already become completable earlier in the month.

The historical pace of console emulation does not look like this.

REA — Reverse Engineer Anything

While the ports and the emulator were making noise, a different tool exploded: REA (morluto/rea).

An MCP server and CLI that lets coding agents (Claude Code, Codex, Cursor, and others) reverse-engineer applications, native binaries, websites, and runtime behavior without source code. It orchestrates existing tools (Ghidra, Hopper, IDA, browser instrumentation) and returns evidence-backed explanations. In roughly two weeks it accumulated hundreds of thousands of stars.

The practical effect is that an agent can now be pointed at a feature and produce a usable understanding plus a path to independent reimplementation. Combined with the clean-room pattern I described in March, the barrier between “I can observe the behavior” and “I can recreate the behavior under a different license or on different hardware” continues to fall.

The Adobe and Office “Craft” ports — imperfect, but the signal is loud

Early October also brought the ArtCraft projects (led by Brandon Thomas): open-source, Rust-based applications described as clean-room reimplementations of major Adobe tools — PhotoCraft (Photoshop), FilmCraft (Premiere), LightCraft (Lightroom), and equivalents for Illustrator, After Effects, InDesign, and Acrobat. Built with heavy assistance from Claude.

A few days later the same effort expanded to Microsoft Office: WordCraft, GridCraft (Excel), and DeckCraft (PowerPoint).

None of these are finished products. The teams themselves rate them as early alpha—feature parity and “ready for real work” numbers typically sit in the 45–60% range depending on the metric and app. File compatibility, edge cases, and polish remain incomplete.

They are imperfect. That is exactly why they matter as signals.

These are public demonstrations that a small team + frontier models can now produce recognizable, runnable, file-compatible approximations of two of the most heavily protected, highest-margin software suites in existence in a matter of weeks. The UI patterns, core workflows, and file formats that previously required years of specialized effort are being reconstructed rapidly enough that the projects are already usable for lighter tasks and improving in public view.

What these stories share

They are not isolated stunts. They are symptoms of the same underlying shift I described in March:

Decompilation and reverse-engineering tooling has matured.
Frontier models are competent enough at low-level and systems code to accelerate the remaining human work dramatically.
Agentic workflows (and tools like REA) make the process more systematic and less dependent on rare specialist knowledge.
The economic cost of “make a working version of X on platform Y” or “understand how Z is implemented so I can rebuild it” has collapsed.

Where is this going?

In the original article I concluded that most software would become effectively open source—by force or by choice—and that the only sustainable thing left to charge for would be convenience. I was thinking primarily about SaaS. I had not fully considered how the same forces would hit other markets.

Now that I am watching it play out in games, I think the game industry is even more exposed.

AAA studios were already struggling with rising costs, risk-averse sequels, live-service fatigue, and audience fragmentation. The ability to rapidly reimplement, port, expand, and redistribute games—especially older or mid-tier titles—removes one of the remaining economic moats: the difficulty of making a working version exist somewhere else. When a decompiled classic can run on N64, Dreamcast, phones, browsers, and PCs with 128-player lobbies within weeks of the tooling maturing, the value of exclusive platform deals, long development cycles, and even some forms of DRM starts to look fragile.

Even games that embrace community content and modding are not automatically safe. Modding communities have historically extended the life of games; the new tools accelerate both the creation of those mods and the independent recreation of the underlying experience itself. The distinction between “official” and “community” versions becomes harder to maintain when the technical barrier to a full alternative keeps falling.

The original prediction still holds, only broader:

Most software will become effectively open source—by force (because protection becomes impractical) or by choice (because fighting it is more expensive than adapting).
The residual value will concentrate in convenience, operations, trust, data network effects, real-time services, and experiences that cannot be trivially recreated offline or offline-first.
Pure expression of code, whether in a SaaS dashboard or a single-player campaign, loses scarcity.

For games this may prove especially brutal. The industry already operates on high fixed costs and hit-driven economics. If the cost of independent recreation and redistribution keeps collapsing, the pressure on traditional publishing and exclusivity models will intensify. Some studios will adapt by doubling down on live services, social layers, competitive integrity, and continuous content that is hard to mirror. Others will not.

Seven months ago I wrote that source code was becoming a common good and that SaaS was mostly dead.

This month Halo became the new Doom, emulators jumped from 1 FPS to 100, reverse-engineering agents went mainstream, and clean-room approximations of Adobe and Office appeared in public.

The direction is clearer than it was in March.

Most software will soon be effectively open source—by force or by choice.

The only remaining question is how many markets realize it in time to adapt.

SDLC != ADLC != ASDLC

Remo H. Jansen — Sat, 17 Oct 2026 23:38:50 +0000

I recently came across this definition:

The Agentic Development Lifecycle (ADLC)—also known as the Agent Software Development Lifecycle—is a software engineering methodology designed for building, testing, deploying, and governing AI agent systems rather than traditional deterministic code.

That sentence made something click. A lot of people (myself included, until recently) have been casually treating ADLC and ASDLC as interchangeable. They’re not.

Once you separate them, the confusion starts to dissolve.

The Three Terms, Cleanly Separated

Term	Full Name	What it is	Primary focus	What is being built?	Who does most of the work?
SDLC	Software Development Lifecycle	The classic methodology for building software	Deterministic systems	Traditional applications & services	Humans (tools assist)
ADLC	Agent / Agentic Development Lifecycle	A specialized lifecycle for creating AI agents	Probabilistic, tool-using, adaptive systems	AI agents and multi-agent systems	Mixed (humans + agents), with heavy emphasis on evaluation & governance
ASDLC	Agentic Software Development Lifecycle	The traditional SDLC performed with AI agents as workers	Any software, delivered agentically	Applications, services, infrastructure… or even agents	Agents execute; humans set intent, guardrails, and approve

Why the Mix-Up Happens

Both terms contain the words “Agentic” and “Development Lifecycle,” so people collapse them. On top of that:

Some writers use “ADLC” loosely to mean “the new way we develop software now that agents exist.”
Others (IBM, Arthur.ai, Glean, Atlan, etc.) use ADLC more precisely: the lifecycle for building agents themselves.
Meanwhile, “Agentic SDLC” / ASDLC has become the clearer label for “we’re using agents to ship regular software.”

The result is a muddy conversation where people argue past each other.

Why the Distinction Actually Matters

If you’re building an agent, you care about different things than if you’re using agents to build a normal product:

Non-determinism and evaluation harnesses become first-class concerns.
Observability, containment, and continuous behavioral monitoring matter more than classic “green tests.”
The “done” state is fuzzier. Agents keep running and adapting after deployment.

Treating that work with a regular (or even agent-assisted) SDLC often leads to brittle agents that look good in demos and then drift or fail in production.

Conversely, if you’re just trying to ship a web app or an internal tool faster with coding agents, you don’t necessarily need the full ADLC machinery. You need a well-governed ASDLC.

Quick Mental Model

SDLC → Building normal software the old way
ADLC → Building agents
ASDLC → Building anything (including agents) with agents

Once you start using the terms this way, a lot of the current discourse becomes much clearer.

Have you noticed the same mixing of ADLC and ASDLC in your circles? I’d love to hear how your team is currently labeling these ideas.

The Agentic Manifesto: What Should the Agile Manifesto Look Like in the Age of AI?

Remo H. Jansen — Sat, 17 Oct 2026 01:40:40 +0000

I've been spending a lot of time thinking about where all the recent advancements in AI are going to take the software industry.

Not just what AI coding agents mean for developers today, or how much faster we can write code, but what happens when these capabilities fundamentally change the economics of building software.

Over the last few months, I've written about several aspects of this transformation:

Each article explores a different part of the same transformation. Together, they've led me to a bigger question.

If AI agents fundamentally change how we build software, should we reconsider the principles that guide how we develop it?

I think the answer is yes.

And I thought it would be interesting to start a conversation about what the Agile Manifesto might look like today, in a world where AI agents are becoming increasingly capable of building software autonomously.

This is an attempt to start that conversation. It's not a declaration that I've figured everything out, and it's certainly not an argument that everything Agile taught us is suddenly wrong.

Quite the opposite. I think many of the original Agile values remain as relevant as ever. But the context in which we apply them is changing.

Why revisit the Agile Manifesto?

The Agile Manifesto was written in 2001, when software development was a very different activity.

Its authors challenged heavyweight processes, excessive documentation, rigid plans, and organisational structures that often got in the way of delivering working software.

They established four values:

Individuals and interactions over processes and tools.
Working software over comprehensive documentation.
Customer collaboration over contract negotiation.
Responding to change over following a plan.

These values helped shape how an entire generation of engineers and organisations approached software development.

But consider what happens when AI agents can implement features, modify existing systems, generate tests, refactor code, and work across entire codebases with increasingly little human intervention.

The bottleneck starts to move.

Writing code becomes cheaper. Producing multiple implementations becomes easier. Experimenting with different approaches becomes more affordable.

But we still need to know whether the software does what we intended, whether it introduces vulnerabilities, whether it behaves correctly in production, and whether it actually solves the customer's problem.

In many cases, the harder problem is no longer producing a solution. It's establishing that the solution is worth trusting.

When code gets cheap, verification becomes expensive.

This is not simply a tooling change. It has consequences for team structures, software architecture, engineering practices, product discovery, and the economics of custom software.

If we can produce software at a rate that exceeds our ability to verify and maintain it, then increasing development capacity alone will not solve our problems.

We need to rethink the system in which software is produced.

From Agile to Agentic

I don't think the answer is to discard Agile and replace it with a methodology that prescribes how to use AI coding agents.

That would miss the point.

The more interesting question is whether the underlying values and principles of software development should evolve to reflect this new reality.

Here are some of the shifts I believe deserve our attention.

1. From producing software to verifying outcomes

Working software was a major improvement over plans and documents as a measure of progress.

But in an agentic world, generating working code is becoming easier. Even passing tests does not necessarily mean we've built the right thing.

We need to focus on verified outcomes: software that demonstrably satisfies its requirements, behaves within its intended constraints, and delivers value to customers.

2. From implicit assumptions to explicit intent

AI agents can operate with considerable autonomy, but they need to understand what they are trying to achieve and what they must not do.

Constraints, specifications, architectural boundaries, acceptance criteria, and executable policies become increasingly valuable because they make intent easier to interpret and verify.

Explicitness isn't the ultimate goal. It's a means of achieving more reliable outcomes while enabling greater autonomy.

3. From supervising implementation to designing autonomous systems

When humans perform most implementation work, we organise teams around people writing, reviewing, and integrating code.

When agents can do much of that work, we need to think differently about the development system itself.

How much autonomy can we safely support? Which decisions require human judgment? Where should verification happen? How do we prevent failures from propagating?

The goal should not be to remove humans from software development. It should be to use human judgment, creativity, and ownership where they matter most.

We may end up with fewer engineers who can accomplish significantly more, but also carry greater responsibility for the systems they create and operate.

4. From optimising implementation to optimising the whole system

The Toyota Production System offers a useful perspective here.

Improving the productivity of an individual step does not necessarily improve the performance of the entire system.

If agents produce code ten times faster but verification, integration, and maintenance remain unchanged, we may simply create a larger queue of work waiting for those activities.

We need to optimise the flow of verified value, not the volume of code generated.

With these ideas in mind, I started wondering what a new manifesto might look like.

The Agentic Manifesto

We are uncovering better ways of developing software by building with intelligent agents and learning from the systems in which they operate.

Through this work, we have come to value:

Verified outcomes over generated output

Explicit intent over implicit assumptions

Human judgment and ownership over human supervision

Safe, continuous flow over controlled, sequential handoffs

That is, while there is value in the items on the right, we value the items on the left more.

Principles behind the Agentic Manifesto

We follow these principles:

1. Our highest priority is to deliver valuable, verified outcomes to customers through early and continuous delivery.

The goal is not to produce more code, complete more tasks, or deploy more agents. It is to solve meaningful problems and establish that we've solved them.

2. Welcome changing requirements and new information. Design systems that make change cheap, verifiable, and reversible.

AI makes experimentation and implementation cheaper. We should take advantage of that flexibility while ensuring that changes can be evaluated and corrected without creating unnecessary risk.

3. Deliver working software frequently, with automated verification integrated throughout development.

Short feedback loops remain essential. Agents should be able to implement, test, evaluate, and refine changes continuously rather than generating large quantities of code that must be reviewed later.

4. Make intent explicit enough for humans and agents to understand what must be achieved and what must never be violated.

Requirements, constraints, invariants, acceptance criteria, and architectural boundaries should be expressed clearly and made executable wherever practical. Explicitness enables autonomy; it does not eliminate the need for judgment.

5. Build verification into the development process, and require evidence proportional to the consequences of failure.

Quality should not depend on hoping that generated code is correct or relying exclusively on human review. Use automated tests, static analysis, security checks, formal methods, runtime monitoring, and independent evaluation where appropriate. Higher-risk changes require stronger evidence.

6. Build quality into the production system rather than relying on inspection at the end.

Detect abnormalities as early as possible, prevent defects from propagating, and stop work when critical verification fails. A system that produces code faster but allows defects to accumulate is not a better system.

7. Give agents the maximum autonomy that can be safely supported by explicit boundaries, reliable feedback, and effective controls.

Autonomy should be earned through evidence and constrained by risk. Agents should be able to make routine decisions independently while respecting permissions, architectural rules, operational limits, and escalation requirements.

8. Preserve human judgment, creativity, and ownership as the capabilities of autonomous systems increase.

Delegate implementation, not responsibility. Humans should focus on defining problems, resolving ambiguity, making consequential decisions, and taking accountability for outcomes. Greater leverage demands greater responsibility.

9. Learn continuously from customers and from the behaviour of software in production.

Specifications and tests cannot anticipate everything. Real usage, operational signals, incidents, and customer feedback must inform both the software and the system that produces it.

10. Prefer simplicity and maintainability over unnecessary complexity, even when implementation is cheap.

The fact that an agent can generate a feature quickly does not mean we should build it. Every additional feature, abstraction, dependency, and architectural decision creates potential costs that extend beyond implementation.

11. Optimise for the flow of verified value, not the utilisation of people, the volume of generated code, or the number of agents deployed.

A productive system minimises unnecessary queues, handoffs, rework, and delays between an idea and a verified outcome. Increasing output without improving the entire system merely moves the bottleneck.

12. Continuously improve the system that produces software.

Every defect, failed verification, production incident, and unexpected agent behaviour is an opportunity to improve our specifications, constraints, tests, architecture, and feedback loops. The production system itself must evolve as our capabilities change.

What does this mean for the future of software development?

I believe AI is changing more than the productivity of individual developers.

It is changing the economics of software itself.

If code becomes a common good, the ability to produce an implementation becomes a weaker competitive advantage. The value shifts towards understanding problems, delivering differentiated outcomes, possessing valuable data and distribution, operating reliable services, and establishing trust.

This could fundamentally change enterprise software.

Organisations may increasingly build and adapt software around their own needs instead of accepting the limitations, complexity, and cost of generic products. Custom software that was previously too expensive to justify could become economically viable.

But this future won't be realised simply by giving everyone access to coding agents.

We need production systems that can support the resulting increase in output without sacrificing correctness, security, maintainability, or human accountability.

That's why I think verification should sit at the centre of an Agentic Manifesto.

Not because every line of code needs a formal proof, or because humans should spend their days supervising machines, but because the ability to produce software is becoming less scarce while the ability to establish that software is correct, valuable, and safe remains essential.

The opportunity is to build systems that let us produce more valuable software with less unnecessary effort, while increasing our confidence in what we deliver.

What do you think?

This is a first attempt, and I'm very interested in hearing what others think.

A few questions I'd particularly like to explore with the community:

Are we approaching a point where the Agile Manifesto needs a successor, or are its original values sufficient for the agentic era?
Is verification the right centre of gravity for a new manifesto, or am I overestimating its importance?
What should happen to team structures and engineering roles when a smaller number of engineers can direct much larger amounts of autonomous work?
If code becomes a commodity, where will the sustainable competitive advantage in software actually come from?
Which of these 12 principles would you change, remove, or add?

I'm especially interested in disagreements and alternative perspectives. I'd rather have these ideas challenged now than present a polished manifesto that fails to address the real problems we will face.

The Agentic Manifesto isn't a finished answer. It's an invitation to work out what software development should become next.

What would you change?

Mutation Testing with TypeScript: How to Verify Your Tests in the Age of AI Coding Agents

Remo H. Jansen — Wed, 14 Oct 2026 22:28:14 +0000

Your tests pass. But would they catch a bug?

Imagine asking an AI coding agent to implement a feature. A few minutes later, it has generated the implementation, written unit tests, and opened a pull request.

The CI pipeline is green. All tests pass. Code coverage looks good.

Can you trust the change?

Not necessarily.

The implementation and its tests may share the same misunderstanding of the requirements. The tests might execute every line without checking the behavior that matters. And a test suite that passes today might not detect a regression tomorrow.

As AI makes code generation faster, we need to think more carefully about how we verify the code we produce.

In this tutorial, we'll explore one technique that can help: mutation testing.

We'll build a small TypeScript project, introduce a bug deliberately, install StrykerJS, and use mutation testing to discover weaknesses in our tests. Then we'll explore how the same technique can help us verify AI-generated pull requests.

By the end, you'll understand not just how mutation testing works, but why it can become an important part of a multi-layer verification strategy.

1. Let's start with a simple problem

Imagine we're building an online shop. One of our business rules is that orders worth £100 or more qualify for free shipping.

Let's implement that rule in TypeScript.

Create a file named src/shipping.ts:

export function calculateShipping(
  orderTotal: number,
): number {
  return orderTotal >= 100 ? 0 : 10;
}

The function is straightforward:

Orders below £100 pay £10 shipping.
Orders worth £100 or more get free shipping.

Now let's write some unit tests.

Create src/shipping.test.ts:

import { describe, expect, it } from 'vitest';
import { calculateShipping } from './shipping';

describe('calculateShipping', () => {
  it('charges shipping for small orders', () => {
    expect(calculateShipping(50)).toBe(10);
  });

  it('offers free shipping for large orders', () => {
    expect(calculateShipping(150)).toBe(0);
  });
});

Run the tests:

npx vitest run

Both tests pass.

Great! We have an implementation, some unit tests, and a green test suite.

But let's ask a slightly different question.

What would happen if somebody accidentally changed >= to >?

Our implementation would become:

export function calculateShipping(
  orderTotal: number,
): number {
  return orderTotal > 100 ? 0 : 10;
}

Now an order worth exactly £100 is charged shipping, even though the requirement says it should qualify for free shipping.

We've introduced a bug.

Run the existing tests again.

They still pass!

Why? Because we tested orders worth £50 and £150, but we never tested the boundary at £100.

Our tests execute the function, but they don't distinguish the correct implementation from this incorrect one.

This is the problem mutation testing helps us investigate.

2. What is mutation testing?

Mutation testing is a technique for evaluating how effectively your tests detect certain defects.

Instead of simply running your tests against your implementation, a mutation-testing tool deliberately makes small changes to your production code and runs the tests against each modified version.

These changes are called mutations. Each modified version is called a mutant.

For example, a mutation-testing tool might make changes such as:

Original code	Mutated code
`a >= b`	`a > b`
`a + b`	`a - b`
`isValid && isActive`	`isValid
{% raw %}`return true`	`return false`

The tool then observes whether your tests detect the change.

There are two particularly important outcomes.

Killed mutant: At least one test fails because of the mutation. Your tests detected the change.

Survived mutant: The tests still pass. Your tests didn't detect the change.

Let's apply this to our shipping example.

The original implementation is:

return orderTotal >= 100 ? 0 : 10;

A mutant changes it to:

return orderTotal > 100 ? 0 : 10;

Our existing tests still pass, so the mutant survives.

That's a useful finding. It tells us that the test suite doesn't distinguish these two implementations.

Notice what we're evaluating: we're not simply asking whether the implementation works for our current examples. We're asking whether the tests can detect selected changes that might introduce defects.

This is a different question from ordinary unit testing and code coverage.

3. Code coverage isn't the same as test effectiveness

You may already measure code coverage in your projects.

Coverage tells you which parts of your code were executed during testing. It's useful, but it doesn't necessarily tell you whether your assertions are strong enough.

Consider this test:

it('calculates shipping', () => {
  calculateShipping(100);
});

The function executes. Depending on the coverage measurement, the test may contribute to branch or line coverage.

But the test doesn't assert the expected result.

The implementation could return £10 instead of £0, and the test would still pass.

Now consider:

it('offers free shipping at exactly £100', () => {
  expect(calculateShipping(100)).toBe(0);
});

This test checks the actual business rule.

It also kills our mutant because the mutated implementation returns £10.

We can summarize the difference:

Code coverage: Did our tests execute this code?
Assertions: Did our tests verify the expected behavior?
Mutation testing: Would our tests detect selected changes to the implementation?

These techniques complement each other. Mutation testing doesn't replace coverage, and a mutation score doesn't prove correctness. Together, however, they can provide more useful evidence about the effectiveness of a test suite.

4. Let's set up StrykerJS

We'll use StrykerJS, a mutation-testing framework for JavaScript and TypeScript.

We'll assume you already have Node.js and npm installed.

If you're following along from scratch, create a project:

mkdir mutation-testing-demo
cd mutation-testing-demo
npm init -y

Install TypeScript and Vitest:

npm install --save-dev typescript vitest

Create the src directory and add the two files from the previous sections.

Now run:

npx vitest run

Make sure your tests pass before continuing.

Initialize StrykerJS

Run:

npx stryker init

Follow the prompts to configure Stryker for your project. Select TypeScript and the Vitest test runner where applicable.

Stryker's setup options can vary with the version and project configuration, so follow the current prompts rather than assuming every project requires an identical configuration.

Official documentation: StrykerJS — Getting Started.

Once initialization is complete, inspect the generated configuration and make sure the mutation scope includes our production file:

src/shipping.ts

Our test file, src/shipping.test.ts, should not be treated as production code to mutate.

You can configure the mutation scope explicitly if necessary. For example, in a TypeScript Stryker configuration, the relevant pattern might be:

mutate: ['src/**/*.ts', '!src/**/*.test.ts'],

Keep the configuration generated by your installed version as the starting point, and adapt it to your repository's conventions.

Run your first mutation test

Execute:

npx stryker run

Stryker will run your tests, generate mutations in the configured production code, and evaluate the results.

The first run may take longer than your ordinary unit tests because Stryker needs to evaluate multiple mutants.

Once it finishes, inspect the terminal output and generated HTML report.

Look for surviving mutants associated with the comparison in calculateShipping.

If the comparison changes from >= to > and all our tests still pass, we've found the weakness we deliberately introduced.

Congratulations: you've just used mutation testing to discover a gap in your test suite.

5. Understand the mutation report

A mutation report contains more than just a score.

The most useful outcomes to understand are:

Killed

A test detected the mutation and failed. This is generally what we want for a meaningful behavioral change.

Survived

The tests passed despite the mutation. Investigate whether this reveals missing test coverage or an assertion that doesn't adequately verify the requirement.

No coverage

The mutated code wasn't exercised by the relevant tests. You may need to investigate whether the code is tested at all.

Timeout

The tool couldn't obtain a normal result within its configured time limit. This doesn't necessarily mean your tests detected the mutation.

Equivalent mutant

The mutation doesn't change observable behavior for the relevant inputs. No meaningful test can distinguish it from the original implementation.

Not every surviving mutant is a defect, and not every mutant needs to be killed. Some mutations are equivalent, some aren't meaningful for your requirements, and some may reveal behavior you don't actually need to constrain.

The purpose of the report is to help you make better decisions about your tests.

What about the mutation score?

Stryker calculates a mutation score based on the results of mutation testing.

A simplified formula is:

Mutation score = killed mutants / valid mutants × 100%

For example, suppose a run produces 100 valid mutants and your tests kill 80 of them. The mutation score is 80%.

That can be a useful signal, but be careful about treating it as a universal quality metric.

Two projects can have the same mutation score while having very different levels of risk. The score depends on the mutations generated, the code being analyzed, and which results are excluded.

A surviving mutation in payment authorization may matter much more than a surviving mutation in an unimportant formatting helper.

Focus on understanding meaningful surviving mutants, not maximizing a percentage at any cost.

6. Improve the tests and run mutation testing again

Let's fix the weakness we discovered.

Our business rule is explicit: orders worth £100 or more qualify for free shipping.

We should test the boundary directly.

Add this test to src/shipping.test.ts:

it('offers free shipping at exactly £100', () => {
  expect(calculateShipping(100)).toBe(0);
});

Run the ordinary test suite:

npx vitest run

The test passes against the correct implementation.

Now run mutation testing again:

npx stryker run

The mutation that changes >= to > should now be killed by the boundary test.

We've improved the test suite because it now detects a meaningful defect.

Notice that we didn't add a test just to improve a score. We added one because the requirement explicitly defines what should happen at the boundary.

This distinction is important.

A mutation-testing report is a source of questions, not an instruction to write more assertions blindly.

For every surviving mutant, ask:

Does this change affect behavior that matters?
What does the requirement say the correct behavior should be?
Should the test suite detect this change?
If so, what is the smallest meaningful test that would detect it?

This approach fits naturally into Test-Driven Development.

TDD helps us implement behavior from requirements. Mutation testing challenges whether our tests can detect selected deviations from that behavior.

7. From local experiments to pull request verification

So far, we've run mutation testing manually.

The next step is to integrate it into CI.

Imagine an AI coding agent creates a PR that modifies calculateShipping and adds several unit tests.

The ordinary CI pipeline runs:

Type-checking.
Linting.
Unit and integration tests.
Other project-specific checks.

Everything passes.

Now Stryker runs against the configured production code. It discovers that the tests don't detect a meaningful mutation in the shipping calculation.

The PR report can expose that weakness before the change is merged.

This is particularly useful for AI-generated code because the implementation and its tests may have been produced by the same agent, based on the same interpretation of the requirements.

The tests passing is evidence that the implementation satisfies the assertions the agent wrote. It isn't independent evidence that those assertions capture every important requirement.

Mutation testing adds another challenge: it deliberately changes the implementation and checks whether the test suite notices.

Should mutation testing block every PR?

I'd start with a gradual approach.

Stage 1: Report findings without blocking merges.

Run mutation testing on a small set of important modules. Learn how long it takes, inspect surviving mutants, and identify noisy or irrelevant results.

Stage 2: Gate critical code.

Introduce stronger requirements for areas where correctness is particularly important, such as permissions, pricing, validation, or state transitions.

Stage 3: Track regressions over time.

Use scheduled runs and historical reports to identify weaknesses across the broader codebase.

For larger projects, Stryker's incremental mode can reuse eligible previous results to reduce the work needed on subsequent runs. See the incremental testing documentation.

One important distinction: mutating changed production files is not the same as testing only the new tests in a PR. You want relevant existing tests to participate in verifying the changed behavior as well.

You should also avoid making a repository-wide mutation score an immediate merge requirement. A small PR shouldn't necessarily be responsible for fixing every weakness in a legacy test suite.

8. Let the AI agent investigate surviving mutants

We can take the workflow a step further.

Instead of using mutation testing only as a CI gate, we can feed its findings back to the AI agent.

Imagine this sequence:

An agent implements a feature and writes unit tests.
CI runs the ordinary checks.
Stryker finds a surviving mutation.
The agent receives the mutation report and relevant requirements.
The agent determines whether the mutation exposes a genuine testing gap.
The agent proposes an additional test or explains why the mutant isn't meaningful.
CI reruns the checks against the revised PR.

For our shipping example, the agent might receive a report showing that changing >= to > doesn't cause any tests to fail.

The agent can then inspect the business rule and recognize that the boundary at £100 is important.

It adds the boundary test, reruns the suite, and verifies that the mutation is now killed.

This creates a feedback loop in which the agent can improve not just the implementation, but also the evidence we have about that implementation.

However, there's a critical caveat.

Don't let the agent blindly optimize the mutation score.

An agent could make a mutant fail by adding an assertion that encodes an incorrect assumption. It could also change production code simply to make the existing tests pass.

The correct behavior must come from explicit requirements, domain rules, or other trustworthy sources—not from the mutation report alone.

The agent can propose a fix, but the final verification should run independently in CI against the revised commit.

9. Mutation testing is one layer in a Swiss cheese model

This brings us to the broader reason mutation testing matters.

No single verification technique catches every defect.

Type checking can detect certain classes of errors, but it doesn't establish that a business rule is correct.

Unit tests can verify specified behavior, but they only check the cases and assertions we've written.

Mutation testing can challenge the effectiveness of those tests, but it can't guarantee that the original requirements were understood correctly.

Integration tests can expose problems between components, but they won't necessarily identify every defect in an individual component.

Code review can identify design and requirement problems, but human reviewers also have limited time and attention.

Each layer has gaps.

This is the idea behind a Swiss cheese model of verification: arrange multiple imperfect defenses so that a defect that passes through one layer has other opportunities to be detected.

A practical model might include:

Layer 1: Types and static analysis

Catch incompatible interfaces, invalid types, and selected classes of programming mistakes.

Layer 2: Unit and integration tests

Check specified behavior, interactions, and component integration.

Layer 3: Mutation testing

Challenge whether tests detect selected changes to the implementation.

Layer 4: Contract and architecture checks

Verify boundaries, invariants, and integration assumptions.

Layer 5: Independent review and production feedback

Identify misunderstood requirements, broader design risks, and failures that automated checks miss.

The layers are complementary. They aren't completely independent: mutation testing relies on the test suite, and both can be based on the same misunderstood requirement.

But combining different forms of verification can provide stronger evidence than relying on any one of them.

This connects directly to the principles discussed in Revisiting the Toyota Production System in the Age of Coding Agents. In particular, the idea of building quality into the production process is relevant here: detect abnormalities as early as possible, rather than allowing them to move downstream.

Mutation testing can be one of the automated checks that helps us detect weaknesses before a pull request is merged.

10. Why this matters when code generation scales

AI changes the economics of software development.

An agent can produce a substantial implementation and its accompanying tests much faster than a human might write them manually.

But generating code is not the same as establishing that the code is correct.

If the volume of changes increases dramatically, manually reviewing every implementation and every test may become increasingly difficult.

We need verification mechanisms that scale with the rate of change.

This is the connection to When Code Gets Cheap, Verification Becomes Expensive.

Mutation testing is not a complete solution to that problem. It has a computational cost, and it only evaluates the mutations it generates. It cannot establish that the implementation satisfies every requirement or that every important defect is represented by a mutation.

What it can do is make one important question cheaper to investigate:

Can our tests detect plausible defects in the code we're producing?

And because the process can be automated, we can apply that challenge repeatedly as code changes.

The objective isn't to maximize code generation or mutation scores in isolation. It's to increase our ability to produce and verify valuable changes without letting the volume of generated code overwhelm our verification capacity.

Conclusion

Mutation testing is a relatively simple idea with important consequences.

We deliberately modify production code, run our tests, and observe whether those tests detect the changes. When meaningful mutants survive, we investigate whether our tests are missing important behavioral checks.

With TypeScript and StrykerJS, it's possible to introduce this technique into an existing project without replacing your test runner or abandoning your current TDD workflow.

And when AI agents generate code and tests, mutation testing provides a useful additional challenge: rather than accepting a green test suite at face value, we can investigate whether it detects selected mistakes.

It doesn't prove that our code is correct. No single testing technique does.

But that's precisely why it belongs in a multi-layer verification strategy.

When code generation scales, verification must scale with it. Mutation testing helps us challenge one of the most important defenses we already have: our tests.

From Building Products to Building Factories: The Industrial Revolution of Software Engineering Is Here

Remo H. Jansen — Mon, 12 Oct 2026 22:48:45 +0000

The Industrial Revolution changed much more than how goods were manufactured. It transformed how people worked, how businesses operated, how cities grew, and how economies created value.

I believe software engineering is approaching a similar transformation.

We are moving from a world where engineers primarily produce software themselves to one where engineers increasingly build and operate systems that produce software for them.

Coding agents are the catalyst, but the real transformation is much bigger than AI writing code. It is about changing the entire system through which software is designed, built, tested, and delivered.

To understand where we might be heading, it helps to look back at the original Industrial Revolution.

What changed during the Industrial Revolution?

Beginning in Britain in the second half of the eighteenth century, the Industrial Revolution transformed an economy dominated by agriculture, hand production, and small workshops into one increasingly organised around machinery, factories, and large-scale manufacturing.

Several fundamental changes drove this transformation.

1. Hand tools became machines.

Skilled craftspeople had traditionally produced goods by hand. New machinery allowed the same kinds of work to be performed faster and at a much greater scale.

2. Small workshops became factories.

Production moved from homes and small workshops into factories where machinery, workers, materials, and processes could be coordinated in one place. The factory became a new unit of production.

3. Human and animal power gave way to new sources of energy.

Steam engines, powered largely by coal, enabled machinery to operate at scales and in locations that had previously been impractical. Later, electricity transformed factories further.

4. Transport networks connected production at scale.

Canals, railways, and steamships made it possible to move raw materials and finished goods more efficiently. Factories could serve markets far beyond their immediate surroundings.

5. Standardisation enabled mass production.

Repeatable processes and increasingly standardised components made production more consistent and scalable. Manufacturing became less dependent on every individual craftsperson completing every step from beginning to end.

6. The organisation of work and the economy changed.

Factories created new roles, management structures, and patterns of employment. Human labour did not disappear, but its nature changed. People increasingly operated, maintained, managed, and improved systems of production.

The important point is that the Industrial Revolution was not simply a story about machines replacing manual labour.

It was a transformation of the entire production system.

And that is where I see a parallel with software engineering today.

The software industry's industrial revolution

Let's map these changes to software engineering.

1. Hand tools → Machines → Coding agents

In traditional software development, engineers are directly responsible for much of the implementation work. They translate requirements into code, write tests, debug failures, and make changes across a codebase.

Tools have always helped. Compilers, IDEs, libraries, frameworks, and CI/CD systems have made engineers more productive. But the engineer has generally remained the primary executor of the work.

Coding agents change that relationship.

An engineer can now delegate implementation, testing, debugging, code exploration, and other tasks to agents. Instead of manually performing every step, the engineer increasingly specifies the desired outcome, provides context, delegates work, and evaluates the results.

The shift is not simply from one coding tool to another.

It is from engineers using tools to produce software towards engineers directing agents that produce software.

2. Small workshops → Factories → Agentic software factories

A traditional software squad resembles a small workshop in some respects. A group of engineers collaborates to design, implement, test, and deliver a product or feature.

Hiring more engineers and creating more squads has historically been a common way to increase development capacity.

But what happens when a significant portion of implementation can be delegated to agents?

We can begin to imagine agentic software factories: production systems that coordinate agents across many tasks, repositories, and workflows.

Rather than scaling only by adding more human teams, organisations could scale by increasing the capacity of their agentic production systems.

A factory might coordinate hundreds or potentially thousands of agents, each working on specialised tasks, while shared infrastructure manages dependencies, execution, testing, verification, and delivery.

This does not mean that thousands of agents will automatically produce thousands of times more useful software. Coordination, context, dependencies, and verification all introduce constraints.

The opportunity is to create a production system that scales without requiring human execution capacity to grow proportionally.

3. Human and animal power → Steam engines → Compute and datacentres

Industrial machinery needed power. Steam engines and, later, electricity provided the energy that allowed factories to operate at scale.

Agentic software production has its own equivalent.

AI agents depend on models, inference, compute infrastructure, memory, storage, and electricity. Datacentres provide much of the physical infrastructure behind this new capability.

This changes the economics of software production.

In a traditional model, increasing implementation capacity often means hiring more engineers. In an agentic model, increasing execution capacity may also mean allocating more compute, using more capable models, and improving the infrastructure that coordinates agents.

Human expertise remains essential, but compute becomes an increasingly important production input.

And just as access to energy helped determine where and how industrial production could scale, access to affordable and reliable compute may influence how agentic software factories are designed and operated.

4. Canals and railways → Orchestration and automated workflows

Factories could not scale in isolation. They needed ways to move materials between production stages and connect suppliers with markets.

Agentic software factories need their own infrastructure for moving work.

Agents must receive tasks, access the right context, use tools, coordinate with other agents, and hand their work to subsequent stages. Their outputs must move through code review, testing, integration, security checks, and deployment.

APIs, task queues, orchestration systems, version control, CI/CD pipelines, and observability become the connective infrastructure of the factory.

The challenge is no longer only how to make an individual agent effective. It is how to move work reliably through the entire production system.

A highly capable agent inside a poorly designed workflow may accomplish very little. A well-designed system can combine specialised capabilities into a reliable process.

5. Craft production → Standardisation and repeatable processes

Mass manufacturing did not emerge simply because factories acquired more machines. Production also became more repeatable.

Agentic software production faces the same challenge.

Agents need clear specifications, stable interfaces, appropriate context, reusable capabilities, and objective ways to evaluate their work.

Without these, adding agents can multiply inconsistent implementations, conflicting changes, and rework.

Standardised workflows and automated evaluations make it easier to delegate work without requiring a human to intervene at every step.

This is also where software architecture becomes particularly important. Good boundaries allow work to be divided into tasks that can be performed and verified independently. Poor boundaries create dependencies that make parallel execution difficult.

The lesson is straightforward: scaling production requires designing the work so that it can be reliably delegated, integrated, and checked.

6. Manual production → Factory operation and management

Perhaps the most important change concerns the role of the engineer.

Industrialisation did not eliminate humans from manufacturing. It changed what they did.

People designed factories, installed machinery, maintained equipment, managed production, controlled quality, and improved processes. Machines performed more of the repetitive physical work, but the system still required human direction and expertise.

The same distinction matters for software.

Engineers will not simply disappear from the production process. Their responsibilities may increasingly move towards designing and operating the systems that produce software.

And this leads to the central idea of this article.

Engineers will build the factories that build software

Consider a modern car factory.

Robots assemble components and perform repeatable operations with speed and precision. But the factory itself was designed by humans. Humans selected and installed the robots, designed the production line, defined its operating procedures, established quality controls, and created the systems that coordinate the process.

Humans also decide which cars to build, how they should work, and what quality standards they must meet.

The robots manufacture the cars. Humans build and operate the system that makes this production possible.

I believe agentic software engineering is moving towards a similar model.

Agents will increasingly perform implementation work, but humans will design the architecture, build the orchestration systems, establish the specifications, create verification mechanisms, and operate the production environment.

The distinction is between building software and building the system that builds software.

These are related, but they are not the same discipline.

In a traditional workflow, an engineer might receive a requirement, design the solution, write the code, test it, fix defects, and deliver the change.

In an agentic workflow, an engineer might instead define the desired outcome, break the problem into verifiable tasks, establish constraints, delegate implementation to agents, and ensure that the resulting changes satisfy the required standards.

The engineer is no longer necessarily the person performing every operation. They become the person responsible for making sure the production system performs those operations correctly.

This does not mean that every engineer will stop coding. Direct implementation will remain valuable, particularly for complex problems, architecture, debugging, and situations where agents cannot reliably complete the work.

But the centre of gravity can shift.

Engineers won't just build software. They'll build the factories that build software.

The engineer becomes a factory architect and operator

If this transformation continues, several capabilities may become increasingly important.

Designing for delegation. Software architecture will need to support work that can be divided into well-defined, relatively independent tasks. Clear boundaries, contracts, and interfaces will help agents work in parallel without constantly conflicting with one another.

Defining intent and specifications. Agents need to know what to build, what constraints to respect, and what success looks like. Ambiguity that a human team might resolve through conversation can become expensive when propagated across many agents.

Building verification systems. Producing code is not the same as producing correct software. Automated tests, static analysis, security checks, integration tests, and higher-level evaluations become essential parts of the production system.

Orchestrating work. Engineers will need to decide how tasks are decomposed, assigned, prioritised, coordinated, and integrated. More agents do not necessarily mean more useful output if they are working on conflicting or poorly defined tasks.

Operating and improving the factory. Engineers will need to understand throughput, bottlenecks, failure rates, rework, latency, compute consumption, and cost. The goal is not to maximise the number of agents running. It is to maximise the amount of valuable, verified software the system can deliver.

Retaining human judgment and accountability. Product decisions, architectural trade-offs, risk acceptance, and responsibility for outcomes cannot simply be delegated away. Automation changes how work is executed, but organisations still need people accountable for what they deliver.

In other words, the engineer's job increasingly becomes designing the conditions under which software can be produced reliably.

Why the Toyota Production System matters

This brings me to a subject I explored in my previous article, Revisiting the Toyota Production System (TPS) in the Age of Coding Agents.

The Toyota Production System is relevant because it focuses on how to organise production, not merely on the capabilities of individual machines.

Two factories can have similar equipment and very different results. The way work flows through the factory, how defects are detected, how quality is maintained, and how the system improves over time all matter.

The same is true of agentic software engineering.

Giving an organisation access to powerful coding agents does not automatically give it an effective software factory.

We need production systems that make it possible to detect failures early, prevent defective work from moving downstream, limit unnecessary work in progress, and continuously improve the process.

Several ideas from manufacturing are especially relevant.

Jidoka: build quality into the process.

In the Toyota Production System, automation is paired with the ability to detect abnormalities and stop when something goes wrong. The aim is not to manufacture defective products faster.

For software agents, the equivalent is to make verification part of the workflow. An agent's output should not automatically be treated as completed work simply because the agent says it has finished. Failed tests, violated constraints, security issues, and integration problems should prevent work from progressing until they are addressed.

Just-in-time production: avoid accumulating unfinished work.

Launching hundreds of agents simultaneously may look impressive, but it can create a large inventory of partially completed changes that compete for attention and integration capacity.

The objective should be to deliver work when it is needed, in manageable increments, while keeping work in progress under control.

Poka-yoke: make mistakes difficult to propagate.

Mistake-proofing means designing a process so that common errors are prevented or caught automatically.

In software factories, this might mean enforcing interface contracts, requiring automated checks, restricting permissions, validating outputs against specifications, and preventing unsafe changes from reaching production.

Kaizen: continuously improve the production system.

Every failed task, rejected change, integration conflict, and repeated agent mistake provides information about the factory.

Rather than treating every failure as an isolated incident, we can use it to improve specifications, tools, tests, workflows, and reusable agent capabilities.

These principles become more important as execution scales. If an agent makes a mistake, the impact may be limited. If a poorly designed workflow allows thousands of agents to repeat that mistake, the problem becomes systemic.

The more automated the factory becomes, the more important it is to engineer the production system itself.

The new bottleneck is not necessarily coding

If agents make implementation cheaper and faster, what becomes scarce?

It depends on the problem, the technology, and the organisation. But several constraints may become more prominent.

Knowing what to build. A factory can produce enormous quantities of output without creating much value if it is producing the wrong things. Product judgment and problem selection remain critical.

Providing clear context. Agents need access to relevant requirements, architectural decisions, existing code, and constraints. Missing or contradictory context can undermine even capable models.

Coordinating parallel work. Some tasks can be performed independently; others have dependencies that limit how much parallelism is useful. More agents can introduce coordination overhead rather than remove it.

Verifying correctness. Generating an implementation may become relatively inexpensive, while proving that it is correct, secure, maintainable, and fit for purpose remains difficult.

Managing cost and resources. Agent execution consumes compute and other resources. A system that generates more code but requires disproportionate verification and rework may not be economically attractive.

This is why I don't think the defining metric of an agentic software factory should be the number of agents it runs or the volume of code it produces.

The more meaningful question is:

How much valuable, reliable software can it deliver, at what cost, and with what level of human intervention?

That is a production question, not merely a coding question.

From engineering capacity to production capacity

Traditionally, organisations have often treated engineering capacity as a function of team size. More developers mean more potential implementation capacity, although coordination costs and other constraints complicate the relationship.

Agentic software factories introduce another possibility.

A relatively small group of engineers could operate a production system that delegates substantial implementation work to agents. The organisation's capacity may depend increasingly on the quality of its architecture, workflows, verification mechanisms, and compute infrastructure.

This does not guarantee that teams will become smaller or that software development will become cheaper in every case. Requirements still need to be understood, complex systems still need to be designed, and generated changes still need to be integrated and maintained.

But it changes the potential economics of building software.

If implementation becomes cheaper, organisations may be able to pursue projects that were previously too expensive, maintain more software, experiment more frequently, or build more specialised products.

At the same time, cheaper production could encourage organisations to generate far more software than they can meaningfully maintain. Without good production controls, lower implementation costs could simply create larger maintenance burdens.

The objective is not to maximise software production at any cost.

It is to make the production of useful software more scalable.

The factory is not the datacentre

One distinction is worth making.

Datacentres provide the compute infrastructure that powers AI models and agents, but they are not the software factory itself.

A software factory is the complete system that turns intent into verified software. It includes the agents, models, tools, codebase, specifications, workflows, orchestration, tests, deployment mechanisms, monitoring, and human operating model.

Two organisations could have access to similar models and compute resources but achieve very different results because their production systems differ.

One might have agents making overlapping changes, producing inconsistent implementations, and relying on humans to discover defects late in the process.

Another might use explicit task boundaries, isolated execution environments, automated verification, controlled integration, and feedback loops to improve reliability.

The difference is not simply the intelligence of the agents.

It is the quality of the factory.

That is why I believe building and operating agentic software factories will become an important engineering discipline in its own right.

The Industrial Revolution is about the system, not the machine

The historical analogy is not exact. Software is not a physical product, AI agents are not industrial robots in every meaningful respect, and software production has different constraints from manufacturing cars or textiles.

Nor does every software organisation need thousands of agents. The right degree of automation will depend on the work being performed, the cost of mistakes, and the economics of the system.

But the underlying parallel is compelling.

The original Industrial Revolution transformed production by combining machines, energy, factories, transport networks, standardisation, and new ways of organising labour.

The emerging transformation in software combines coding agents, compute infrastructure, orchestration, reusable capabilities, automated verification, and new ways of organising engineering work.

The breakthrough is not any one component. It is how the components work together.

And the human role remains central, although it may change substantially.

Humans designed the factories that allowed machines to manufacture goods at scale. In an agentic software world, engineers will increasingly design, build, operate, and improve the factories that allow agents to produce software at scale.

We should therefore spend less time thinking only about how to make an individual coding agent more productive and more time thinking about how to build a production system that reliably turns ideas into working software.

The future of software engineering may not be about having more engineers writing code. It may be about having more engineers building better systems for producing software.

The Industrial Revolution of Software Is Here.

Building Agentic Engineering Resilience: Using VS Code Copilot Automations + Datadog MCP for Recurrent Error Detection and Fixing

Remo H. Jansen — Sat, 10 Oct 2026 11:57:49 +0000

In the agentic engineering world, coding agents generate a lot of code quickly. The bottleneck shifts from writing code to verifying it. I’ve written about this shift before — the need for explicit systems and multilayer verification (see my earlier posts on dev.to/remojansen, especially When Code Gets Cheap, Verification Becomes Expensive).

This post documents one concrete, useful layer of verification you can build today: a scheduled automation that periodically hunts for recurrent 4xx/5xx errors (or high-error-rate endpoints), performs root-cause analysis, and attempts a TDD-style fix. Over time this compounds into greater system resilience.

We’ll use two recent capabilities:

Automations (Preview) in VS Code + GitHub Copilot
The Datadog MCP Server so the agent can query real observability data

1. Automations (Preview) in VS Code GitHub Copilot

Automations let you save an agent task (prompt + session configuration) and run it on demand or on a recurring schedule (hourly, daily, or weekly). Routine work no longer has to be started manually.

From the VS Code 1.137 release notes:

Enable the setting chat.automations.enabled.
Open the Agents window and select Automations in the sidebar.
Start from a template (catch up on changes, triage issues, find bugs) or write your own prompt and schedule.
Run on demand first to review behavior, then enable the schedule.
Automations are in Preview and rolling out gradually (off by default on Stable, on by default in Insiders).

How to enable and create one

Open Settings and set chat.automations.enabled to true.
Open the Agents window → Automations → Create Automation (or pick a template).
Give it a clear name and a precise prompt that states the task, scope, and expected output.
Choose workspace, agent, model, and permission level.
Set Schedule to Manual first, create it, run it with Run now, review the session, then edit and switch to Weekly (or Daily) once you’re happy.

Important operational notes from the official docs:

The machine / Agent Host must be available for scheduled runs.
An automation runs one session at a time.
Review the agent’s permission level carefully before enabling unattended schedules.

Official documentation: Create and manage agent automations

2. Enabling the Datadog MCP Server in VS Code

The Datadog MCP Server gives the agent structured access to logs, APM, metrics, error tracking, etc.

Recommended path (from the official Datadog docs):

Install the official Datadog extension for VS Code.
Sign in to your Datadog account inside the extension.
Run the command Datadog: Open MCP Configuration Assistant and follow the guided setup.

Alternatively (manual):

Open the command palette → MCP: Open User Configuration (or edit .vscode/mcp.json / user MCP config).
Add a server entry pointing at the Datadog MCP endpoint for your site, e.g.:

{
  "servers": {
    "datadog": {
      "type": "http",
      "url": "https://mcp.datadoghq.com/v1/mcp?toolsets=apm,error-tracking,logs"
    }
  }
}

(Use the correct regional endpoint for your Datadog site. Replace or expand the toolsets query parameter as needed.)

After configuration, start the server and complete the OAuth flow when prompted. Confirm the tools appear in the agent’s tool picker.

Critical warning: the 128-tool limit

GitHub Copilot (and the underlying models) enforce a hard limit of 128 tools per request. Many MCP servers (including Datadog when all toolsets are enabled, plus other MCPs you may have) easily exceed this. When the limit is hit you get errors such as “You may not include more than 128 tools in your request” and other tools become unavailable.

Best practice: load only the toolsets you actually need by using the toolsets query parameter in the MCP URL (or the equivalent filtering mechanism). Example toolsets relevant to error hunting: apm, error-tracking, logs. Avoid toolsets=all unless you are certain the total stays under the limit or you use virtual tools / selective enabling in the tools picker.

You can also deselect tools or entire MCP servers in the Chat / Agent tools picker for a given session.

3. A Verification Layer for Recurrent Errors

Agents produce a lot of code; the systems that stay healthy are the ones that continuously look for the same classes of failure and close the loop with a test + fix.

Here’s a practical pattern:

Skills (slash commands / reusable prompts) you can develop

/find-top-recurrent-errors

Query Datadog (via MCP) for the top 5 repeating 4xx/5xx errors or the endpoints with the highest error rates over the last 7 days. Surface frequency, sample traces/logs, and affected services.
/root-cause-analysis

Given the error data, dig into traces, logs, recent deployments, and code paths to propose a concrete root cause.
/tdd-bug-fix

Write a failing test that reproduces the root cause, then implement the minimal fix that makes the test pass. Prefer small, reviewable changes.

Putting it together in an Automation

Create an automation that runs, for example, every Monday morning:

You are a reliability-focused agent.

1. Use the Datadog MCP tools to identify the top 5 recurrent 4xx/5xx errors (or endpoints with the highest error rates) from the past 7 days.
2. For each, run a focused root-cause analysis using available traces, logs, and code context.
3. For the highest-impact issue that looks fixable in this codebase, follow a TDD approach:
   - Write a test that reproduces the failure.
   - Implement the smallest correct fix.
   - Ensure the test passes and no obvious regressions are introduced.
4. Summarize findings, the proposed fix, and any remaining risks. Do not push or merge; leave changes for human review.

Prefer precision over volume. If data is insufficient, say so clearly.

Schedule it weekly (or daily if your error volume is high). Start with Manual runs until the prompt and permissions feel solid.

Over successive weeks the automation surfaces the same recurring problems, the skills improve, and the codebase gradually becomes more resilient because the verification loop is explicit and repeated.

Why this matters

When code generation is cheap, the expensive part is making sure the system keeps working under real traffic. A scheduled agent that looks at production error patterns, reasons about root cause, and produces a test + fix is one concrete multilayer verification practice you can put in place today.

It is not a replacement for human review, observability culture, or proper incident process — it is an additional, automated layer that compounds over time.

Try the automation with a conservative permission level and a carefully scoped toolset first. Iterate on the prompt and the skills. The goal is not fully autonomous fixing; it is reliable, repeatable detection and a high-quality starting point for a human engineer.

Happy (and more resilient) coding.

From T-Shirt Sizes to Token Quotes?

Remo H. Jansen — Sat, 10 Oct 2026 02:11:08 +0000

I'll start with my position, because the rest of the article depends on it.

I'm against estimation. I have been for a long time. My team has already stopped estimating and stopped doing sprint planning, and I have no plans to bring either back. One thing is left: when upper management asks how big something is, we still answer with a t-shirt size.

Even that has started to feel wrong.

In an earlier article I argued that coding agents are finally killing Scrum. In another I argued that when code gets cheap, verification gets expensive. This article follows from both. If teams no longer estimate, and producing code is no longer the constraint, what should we say when the organisation still asks "how big is this?"

I'm not going to try to bring estimation back to teams. My question is narrower: if an organisation still needs a size signal, what should that signal be?

My hypothesis is that token quotes could replace t-shirt sizes.

I want to be careful about how much confidence that deserves. This is a hypothesis, not a finding. I don't have a dataset of token quotes compared with actual results, and I haven't run the experiment. Some of what follows rests on arguments I've made before and still stand by, mainly that verification has become the bottleneck. Some of it is speculation about a metric nobody has tested yet, including me. I'll try to keep the two apart, and I've included a section on how to test the idea and what would show it to be wrong. Any numbers in this article are illustrative.

I should also be clear about context. I'm describing a team that does most of its implementation work with coding agents, in an organisation that tolerates working without sprint commitments. Plenty of teams don't work like that, and I come back to where this does and doesn't apply near the end.

Estimation never really worked

We all know the problems.

The planning fallacy says we consistently underestimate how long things will take, even when we know about the planning fallacy. Hofstadter's Law says it always takes longer than you expect, even when you take Hofstadter's Law into account. Decades of software projects have confirmed both.

Story points and t-shirt sizes were meant to fix this by moving away from time. Size work relative to other work, the reasoning went, and stop pretending you can predict hours. Then velocity came along and turned points back into time. "We do 30 points a sprint, so this epic will take four sprints." Relative estimation became absolute estimation again with an extra step.

The #NoEstimates movement was right in principle. But as I argued in the Scrum article, organisations lean toward practices they can follow over principles they have to internalise. "Don't estimate" is a principle. Organisations still need something to put in the box, so the box got filled with points and t-shirts.

It's worth asking what management actually needs from that box. Sometimes it really is a date: a contract, a regulatory deadline, a launch tied to a marketing campaign. This article has nothing useful to say about those cases. But in my experience, most of the time the request for a date is standing in for something else. What they need is a relative size signal. Is this bigger or smaller than that? By roughly how much? Which items are outliers we should discuss before committing? They use it to prioritise, to sequence, and to notice when something is much larger than expected.

A size signal is a reasonable thing to want. The problem is how we've been producing it.

Why t-shirt sizes feel wrong now

A t-shirt size is a human's intuition about how much human effort a piece of work will take. Someone who knows the codebase looks at a ticket, compares it with past work, and says "that's an M."

That made sense when human implementation effort was what drove delivery. It no longer is.

With coding agents, implementation time collapses, and it also gets noisy. The same ticket might take an agent four minutes or forty, depending on how much context it needs to find, how many dead ends it explores, and how clearly the intent was expressed. Wall-clock time stops being a useful signal.

More importantly, the constraint has moved. As I argued in When Code Gets Cheap, Verification Becomes Expensive, producing code and establishing that it is correct are two different activities. Agents have made the first dramatically cheaper. They haven't done the same for the second. Or, in the terms of my TPS article, production now outpaces inspection.

So a t-shirt size is now partly an intuition about a quantity that matters less than it used to. (Not entirely: a good t-shirt size often quietly includes review and risk, and that's a point I'll come back to, because token quotes don't.) It still costs real team time to produce: someone has to read the ticket, think about it, and agree with whoever else is in the room. And it can never be checked. Nobody can measure whether something "really was" an L. There's no feedback loop, so the estimates never get better.

A signal that costs human time, increasingly measures the wrong thing, and can't improve looks like a bad deal. The question is whether there's a better one.

The token quote

A token quote is an estimated range of tokens attached to an issue or ticket before work starts. An agent generates it in a pre-flight step: it reads the ticket, explores the relevant parts of the repository, and returns a range along with its reasoning. No code is written.

The output might look like this (illustrative):

Token quote: 180K (P50) – 420K (P90)
Reasoning: Touches the billing module and two API contracts.
           Billing has strong types and a schema; the notification
           service it integrates with does not, and will need exploration.
Confidence: medium. The acceptance criteria don't specify retry behaviour.

That's all there is to it. The useful part is what this signal is and isn't.

Why tokens and not dollars

When I first thought about this idea, the obvious move was to convert tokens into money. Tokens have a price, so why not report "this ticket will cost about $20"?

I think that would be a mistake.

A dollar figure reads as the cost of the work. If management sees $20 next to a ticket, they'll anchor on $20. They'll forget the engineer who reviews the change, the product owner who validates it, the time spent on integration, coordination and deployment, and the operational cost of running it afterwards. The token cost is a small fraction of the total cost of a change. Showing it in currency invites exactly the wrong conclusion: that software has become almost free to build.

Tokens avoid that trap because they're deliberately abstract. "1.5M tokens" doesn't look like a budget line, and nobody will mistake it for the cost of a feature. But anyone can see that 1.5M tokens is much bigger than 10K tokens. The unit works as a relative size signal without pretending to be anything else.

That is exactly what t-shirt sizes were trying to be. The aim is for token quotes to keep the useful property (relative size) and drop the misleading one (a false sense of precision about cost). Whether they really capture the right relative size is the open question I come back to below.

Why token quotes might be better than t-shirt sizes

They cost the team nothing. An agent produces the quote. There's no meeting, no planning poker, no refinement session. This is the main reason the idea doesn't bother me as someone who is against estimation: no human time goes into it. It's a by-product, not a ceremony.

They can be checked against reality. This is the property t-shirt sizes and story points never had. Issue trackers already record when work starts and when it's done, so every quote can be compared with the ticket's actual cycle time, with nobody filling in a timesheet. For the first time, a size signal can be checked against an outcome that was measured, not one somebody estimated.

They're on a continuous scale. A t-shirt size has five buckets. A token quote can tell an outlier apart from something that is merely large.

If management is attached to the familiar vocabulary, the labels can be kept and derived from token bands instead of gut feel. For example (illustrative thresholds only):

Label	Token quote (P50)
S	< 50K
M	50K – 250K
L	250K – 1M
XL	≥ 1M

The labels stay. The meeting that produced them goes.

The assumption everything rests on

All of this depends on one assumption I can't yet back up: that a token quote is a meaningful proxy for the size management cares about. Below, I'll pin "size" down as the ticket's end-to-end cycle time.

There are good reasons to doubt it. Tokens measure how much work the agent did. They don't measure how much work the humans will do. A one-line change to an authentication flow might cost a few thousand tokens and still need a careful security review. A large, mechanical rename across a codebase might cost a million tokens and need almost no review. If management reads token quotes as "how big is this for us", those two cases will mislead them in opposite directions.

A t-shirt size from an experienced engineer usually includes that risk and review effort, even if nobody says so. A token quote doesn't. That's a real loss, and it's the strongest argument against the whole idea.

My guess is that across a typical backlog, token quotes and end-to-end cycle time are correlated well enough to be useful as a relative signal, with the exceptions being exactly the high-risk, low-volume changes that should get human attention anyway. But that is a guess. It's the first thing to test, and if it turns out to be wrong, the token quote is a poor replacement for a t-shirt size.

Calibrating against the right thing

At first, the quote is just the agent's guess. I don't know how good that guess will be. The agent has one advantage over a human gut feel: it actually reads the code it would need to change. It also has clear disadvantages: it lacks the organisational context, history and tacit knowledge an experienced engineer brings, and models are not known for being well calibrated about their own uncertainty. The one thing in its favour that isn't in doubt is that it's free.

The obvious way to improve the guess would be to compare quoted tokens with actual tokens, then eventually train a model on (ticket → actual tokens). That would probably work, in the narrow sense that the model would get good at predicting token consumption. But it would answer the wrong question. A perfectly calibrated prediction of how many tokens an agent will spend tells you nothing about whether token consumption is what management should care about. You'd be validating the measuring instrument, not what it measures.

The comparison that matters is between the token quote and the issue's cycle time: how long the ticket actually took from starting work to being done, including review, rework, waiting and verification. Cycle time is the outcome the size signal is supposed to stand in for. If tokens and cycle time move together, the quote is a useful relative size signal. If they don't, it isn't, however accurately it predicts its own token usage.

There's an apparent irony here: I'm against estimating time, and I'm proposing to validate against time. The difference is that cycle time is measured, not forecast. Nobody is asked to predict it. It's recorded by the tracker as a side effect of the work, and it captures exactly what the token count misses, which is the human side: review, verification, back-and-forth on requirements.

Cycle time is a noisy target, though. It includes queueing, reprioritisation, people on holiday, and tickets left blocked over a weekend. Using it well needs a consistent definition (for example, from "in progress" to "deployed"), and possibly excluding time spent explicitly blocked. Even then, expect a correlation, not a precise prediction.

Actual token usage is still worth recording, but as a diagnostic, not as the validation target. A large gap between quoted and actual tokens tells you something about the agent's pre-flight step. It doesn't tell you whether the quote means anything.

The long-term step, and I want to be clear that this is aspirational, would be to train your own model on historical (token quote + ticket features → cycle time) data. Plausible features include the modules touched, how explicit those modules are, how clear the ticket is, whether a prototype is attached, and which model or agent did the work. At that point the token quote stops being the answer and becomes one input among several for predicting relative size. I don't know how well this would work. It's a direction, not a recipe.

If the approach works at all, it might produce signals that are more useful than the number itself. These are also untested.

Going over the quote is a reason to reassess. When an agent goes well past its P90, something may be wrong: an ambiguous requirement, missing context, or a hidden dependency. In the TPS article I described this kind of signal as an Andon cord: it draws attention to the problem, which isn't the same as shutting the line down. Some tasks legitimately go over, especially exploratory ones where the point is to find out what's there. So a breach should trigger reassessment, not an automatic stop. The agent pauses at a sensible checkpoint and summarises what it has found and why the work is bigger than expected. A human then decides whether to continue with a revised quote, split the ticket, clarify the requirement, or abandon the approach. Tickets that are openly exploratory can be marked as such and given a wider range from the start. If there's a hard cap at all, it should be set well above the P90 as protection against runaway loops, not used as the main control.

Explicitness becomes measurable. In From Rigidity to Explicitness I argued that constraints such as types, schemas and contracts act as context compression. They tell the agent what the system means, so it doesn't have to infer it. Implicit codebases force agents to spend tokens rediscovering intent. Tokens per change, broken down by module, could become an indicator of where a codebase is hard to work in: a kind of tech debt you can put a number on. Interpreting it would need care, though. Some modules are expensive because they're messy, and others because the domain is genuinely complex.

Spec quality becomes visible. Clear tickets, and especially tickets that come with a working prototype, should produce tighter quotes than vague ones. The cost of ambiguity becomes something you can see.

One caveat on comparisons: token counts are not directly comparable across models. Tokenisers differ, and so does how efficiently models reason. Historical comparisons need to record which model produced the numbers, or be normalised to a reference model.

Don't estimate verification. Invest in reducing it.

This is where I want to put most of the weight. It's also the part of the article that doesn't depend on token quotes at all. If the token-quote idea turns out to be wrong, the argument in this section still holds, because it rests on my earlier articles, not on an untested metric.

Once you accept that tokens are only part of the cost of a change, the obvious next step is to estimate the rest. If tokens size the machine side, why not add a verification estimate for the human side: reviewer-hours, risk classes, approval tiers?

I think that would be a mistake. It would bring planning poker back, aimed at a quantity that is even harder to predict than implementation effort. How long it takes to establish that a change is correct depends on things that are hard to know in advance: what the reviewer finds, which edge cases turn up, how the change interacts with everything around it.

More fundamentally, estimating verification only predicts the cost of the bottleneck. Automating verification reduces it.

I want to be precise about how far that goes. Automation doesn't eliminate verification. It takes over the parts that can be made mechanical: type errors, contract violations, regressions in covered behaviour, invalid states the system can't represent. A substantial part of verification can't be handed over that way. Someone still has to judge whether the requirement was the right one, whether the system behaves the way users actually need, whether a change opens a security hole no test was written for, and whether there are operational, legal or product risks that never show up in a pipeline. Those judgements remain human, and in an agentic workflow they become a larger share of the remaining work.

The point of automation isn't to remove humans from verification. It's to stop spending scarce human attention on the mechanical checks, so it's available for the judgements only humans can make.

That difference matters more than it might seem. As I argued in the verification article, implementation cost is mostly paid once, but verification cost is recurrent. We pay it when we build a feature, and again whenever we change it, refactor it, migrate its data, upgrade its dependencies, or let an agent modify it. Any investment that lowers the cost of verification keeps paying off on every future change. Effort spent estimating verification has to be spent again for every ticket, and it doesn't make the next change any cheaper to verify.

So, for teams in a position like mine, I'd suggest that the time that used to go into estimation is better spent on making verification scale. I'm not claiming this is right for every team. If your organisation genuinely depends on forecasts, you may not be able to free that time up. But where you can, this is the trade I'd make. In practice, that means:

Making invalid states unrepresentable. Strong types, schemas, database constraints and explicit state machines turn some classes of bugs into things the system cannot express, so those particular mistakes don't need checking again.
Contracts at boundaries. Contract tests between services and modules mean integration correctness is checked mechanically rather than rediscovered in review.
A fast, tiered verification pipeline. Cheap mistakes should be caught cheaply, by parsers, type checkers and static analysis, before expensive tests or human attention are needed.
Prototypes as requirements. As I argued in the Scrum article, a working prototype from the person who understands the problem validates intent before the build starts. That reduces, though it doesn't remove, the "is this what you meant?" verification later.

None of this needs a forecast. All of it makes the mechanical part of verification cheaper and faster, for humans and for agents, and leaves more human attention for the part that can't be automated.

Bigger changes cost more to verify, now and later

There's one relationship between size and verification worth stating explicitly, though it doesn't need its own estimate.

A large change is harder to verify today: there's more to review, more interactions to consider, and more ways for something subtle to slip through. It also leaves a larger surface of behaviour that has to be re-verified whenever related code changes. Size adds to verification cost twice: once now, and again with every future change nearby.

That gives the token quote a second possible use. A large quote isn't a reason to estimate more carefully. It's a prompt to split the work into smaller changes, each of which is cheaper to verify and leaves less behind to re-verify. The quote doesn't need to predict verification cost to be useful here. It only needs to flag size early, while splitting is still cheap, with the caveat from earlier that a small quote doesn't mean a change is low-risk.

How to test this, and what would prove it wrong

Because the idea is untested, the honest way to adopt it is as an experiment, not a policy. Fortunately it's a cheap experiment. Token quotes cost no team time, so they can run alongside existing t-shirt sizes for a few months without changing anything else.

Over that period, I'd want to answer four questions:

Do token quotes track cycle time? Is there a useful correlation between the quote and the ticket's measured cycle time? Do tickets in a higher token band reliably take longer end to end? This is the key assumption from earlier and the question that matters most. If the correlation is weak, token quotes are measuring agent effort and nothing management needs.
Where do they diverge? Look at the tickets where quote and cycle time disagree most. If the outliers are mostly high-risk, review-heavy changes, as I'd expect, that's a known limitation that can be flagged. If they're random, the signal is noise.
Do they agree with experienced engineers? Run t-shirt sizes and token quotes side by side, and check which one correlates better with cycle time. If the t-shirt size usually wins, the human intuition is capturing something the tokens miss.
Do they change decisions? Did management prioritise, sequence or split work any differently because of the quotes? A signal nobody acts on isn't worth producing, however cheap it is.

Notably absent from this list: "does the agent predict its own token usage accurately?" That's worth monitoring, but a quote can be perfectly calibrated against tokens and still useless as a size signal.

If the answers come back mostly negative, the conclusion should be to keep t-shirt sizes, or to drop sizing altogether, rather than to keep adjusting the token quote until it looks useful.

Caveats

There are ways this can go wrong, and they're worth naming.

Goodhart's Law. If teams are judged on how well they match their token quotes, the quotes will stop being informative. They're a sizing signal for management, not a performance metric. The moment they become a target, they become story points again.

Tokens are not cost. I'll repeat this because it's the easiest mistake to make. A token quote says nothing about labour, review, coordination or operations. It shouldn't be converted to currency for reporting, however tempting that is.

Non-determinism. The same ticket, run twice, will use different numbers of tokens. That's why a quote is a range, not a point.

Model drift. New models change token efficiency, sometimes dramatically. Historical baselines need to be revisited when the underlying model changes.

No new ceremony. If token quotes ever turn into a meeting, with people debating whether a ticket is really 200K or 400K, the approach has failed. The whole point is that no human time goes into producing them.

Where this doesn't apply

The argument assumes a particular setting, and it's worth being explicit about where that setting doesn't hold.

Real external deadlines. Fixed-price contracts, regulatory dates and coordinated launches need forecasts in time. Token quotes don't provide those, and they aren't meant to.
Heavy cross-team dependencies. When several teams have to sequence work against each other, some form of time-based planning is often unavoidable.
Teams where agents do little of the work. If most implementation is still done by hand, human effort is still the main driver, and a t-shirt size may well remain the better signal.
High-risk domains. Where most changes need careful human review whatever their size, such as safety-critical, financial or security-sensitive systems, the gap between agent effort and total effort is at its widest. Token quotes would be at their least informative exactly where the stakes are highest.

Outside those cases, I think the idea is worth trying. Inside them, I'd be cautious.

Summary

There are two separate claims in this article, and they deserve different levels of confidence.

The first is one I'm fairly confident about, because it builds on arguments I've made before. In agentic development, verification is the constraint. Effort that makes verification cheaper keeps paying off on every future change, and effort spent forecasting it doesn't. Where a team has the freedom to choose, I'd put the time into verification automation rather than estimation.

The second is a hypothesis. If an organisation still needs a size signal, agent-generated token quotes might be a better one than t-shirt sizes. They cost the team nothing, they can be checked against measured cycle time, and expressing them in tokens rather than dollars avoids presenting a partial cost as the whole cost. Whether they actually track the size that matters is an open question, and the way to answer it is to run them alongside existing sizes and look at the data.

Conclusion

In my experience, when management asks for estimates, what it most often needs is a sense of relative size, so it can prioritise and spot outliers. We've met that need with rituals that cost team time, measure human effort, and can never be checked.

Token quotes could meet the same need at no cost to the team, in a unit that can be measured against reality. I think they're worth an experiment. I don't yet know whether they'll survive one.

Either way, they only answer the easy question. The hard question in agentic development isn't "how big is this?" It's "when can we trust it?" That one can't be answered by estimating harder. It's answered by making verification cheaper.

If you need a size, try tokens, and measure whether they work. If you need speed, invest in verification. Neither should require an estimation meeting.

When Code Gets Cheap, Verification Becomes Expensive: How AI changes the economics of software architecture

Remo H. Jansen — Mon, 28 Sep 2026 21:52:55 +0000

Recently, I was having a conversation at work about how we should implement a feature. I was advocating for one particular approach, and one of the reasons I preferred it was that I thought it would be easier for AI agents to work with. The structure was more explicit, the possible states were easier to reason about, and it would give us better opportunities to automate verification and put guardrails around future changes.

One of the stakeholders pushed back with a very reasonable point: we shouldn't design software a certain way simply because it is easier for AI agents. We should design it based on what is best for our users.

I agree completely.

But the conversation made me realise that there is an important part of my reasoning that I hadn't articulated well. I wasn't putting the AI agent ahead of the user. I was still trying to find the solution that gave us the best combination of user experience, reliability, simplicity, time to market, implementation cost, operational cost, and long-term maintainability. The fact that an agent could understand and verify the resulting system more easily was one of the variables in that equation, not the objective itself.

That distinction matters because I think AI is changing the economics of software architecture in a way that we haven't fully absorbed yet.

The cost of writing software is changing

For decades, one of the biggest constraints in software development was the cost of producing software. Developers had to understand the problem, design the solution, write the code, review it, test it, debug it, and maintain it. Engineering capacity was scarce, so reducing the amount of implementation work required was extremely valuable.

AI coding agents are changing this equation. They can generate implementations, write tests, refactor code, explore unfamiliar codebases, migrate APIs, and perform many other tasks at a speed that would have been difficult to achieve with humans alone.

But there is an important asymmetry here: generating code is not the same thing as establishing that the code is correct.

An agent can produce a thousand lines of code in minutes. That doesn't mean we can establish in minutes that those thousand lines correctly implement the desired behaviour, don't introduce regressions, preserve important invariants, and interact correctly with the rest of the system.

This means that as the marginal cost of producing code decreases, another cost becomes relatively more important: the cost of verification.

And verification isn't something we pay for once.

We verify software when we initially build it. We verify it when we change it. We verify it when we refactor it. We verify it when dependencies change. We verify it when another feature interacts with it. We verify it when an AI agent modifies it. We verify it when a bug is fixed. We verify it when we migrate data. We verify it when we upgrade infrastructure.

Implementation costs are often largely one-off. Verification costs are recurrent.

That changes the architectural equation.

Implementation cost is not the whole cost

Imagine that we have two possible implementations for the same feature.

Solution A costs $100,000 to implement, but because the architecture doesn't provide strong guarantees, every subsequent change requires significant testing, review, and regression checking.

Solution B costs $120,000 to implement, but the architecture makes many invalid states impossible and gives us strong automated guarantees about the behaviour of the system.

If we only look at implementation cost, Solution A appears cheaper.

But software doesn't stop costing money when the initial implementation is complete.

If Solution A requires an additional $20,000 every year in verification and regression work while Solution B requires only $5,000, then the initial $20,000 saving disappears very quickly. Over the lifetime of the system, the more expensive implementation can become the cheaper system.

The exact numbers aren't important. The important thing is that verification is recurrent.

This is why I think we need to start treating verifiability as an architectural concern.

When comparing two architectures, we shouldn't only ask how much they cost to build. We should also ask how cheaply and reliably we can establish that they are correct over the lifetime of the system.

Verification isn't just testing

When I talk about verification, I don't mean that we should simply write more tests.

Tests are extremely useful, but there is a more powerful form of verification: designing the system so that certain classes of incorrect behaviour cannot happen in the first place.

There is a huge difference between saying, "We have tests that check this invalid state doesn't happen" and saying, "This invalid state cannot be represented by the system."

The latter is an architectural property.

Strong types are a simple example. If a function accepts a UserId rather than an arbitrary string, the type system can eliminate an entire class of mistakes before the program runs. Database constraints can prevent invalid relationships. State machines can restrict which transitions are possible. Schemas can prevent malformed data from entering a system. Idempotency can eliminate entire classes of duplicate-operation bugs. Explicit boundaries can reduce the number of possible interactions between components.

In each case, we are moving some of the burden of verification away from humans and tests and into the structure of the system itself.

The system becomes easier to verify because the system itself provides guarantees.

A database can become part of the state machine

Consider a workflow where a piece of data moves through several states.

Perhaps it starts as Created, then becomes Validated, then Processed, and finally Completed.

One way to implement this is with a single event table:

WorkflowEvents

id
workflow_id
event_type
payload
created_at

The application then needs to determine whether a particular event is valid given the previous state. The code needs to understand which transitions are allowed, whether the previous event exists, whether the payload is valid for the event type, and whether the workflow has reached an impossible state.

We can write tests for all of this. We can write application-level validation. We can ask an AI agent to reason about the state machine. We can review the implementation.

Another approach is to model the workflow more explicitly in the persistence layer. We could have event-specific structures where each stage references the previous stage through a foreign key, with schemas specific to that event.

Conceptually:

Created
   ↓
Validated
   ↓
Processed
   ↓
Completed

Validated cannot exist without a corresponding Created. Processed cannot exist without the appropriate Validated record. Completed cannot exist without the preceding state.

Each event can also have a schema that expresses the data required for that particular stage.

Now the database isn't just storing the state of the workflow. It is helping enforce the workflow.

We have effectively moved part of the finite-state machine into the persistence model.

I'm not suggesting that this is always the right design. It introduces its own complexity and trade-offs, and there are many workflows where a simpler event table or a different state-machine implementation would be preferable. The important point is not that one particular architecture is universally superior.

The important point is that the architectural choice changes the cost of verification.

With the first design, we may need to repeatedly establish through application logic and tests that invalid transitions cannot occur.

With the second design, some invalid transitions simply cannot be persisted.

That is a very different property.

Make invalid states unrepresentable

This is an old idea in software engineering, but AI makes it increasingly valuable.

If we can make an invalid state unrepresentable, we don't need to repeatedly verify that nobody has accidentally created that state.

Consider the difference between these two statements:

"We have tests proving that this state should never happen."

and:

"The architecture makes this state impossible."

The second statement is much more powerful.

It means that a future developer doesn't need to remember the rule. A future refactor doesn't need to rediscover the rule. An AI agent doesn't need to infer the rule from a collection of tests and documentation. A code reviewer doesn't need to notice a subtle violation.

The constraint is part of the system.

This is what I mean when I talk about verification becoming an architectural concern. Architecture can determine not only what the system is capable of doing, but also what incorrect things the system is capable of doing.

AI makes these properties more valuable

This is where AI agents enter the picture, but I don't think the argument should be "design software for AI."

The better argument is that AI increases the rate at which software can change.

If the number of changes increases, the cost of verifying those changes becomes more important.

Imagine two systems. In the first, every change requires a human to reconstruct a complicated mental model of the system and determine whether a subtle invariant has been preserved. In the second, the architecture expresses many of those invariants through types, schemas, constraints, contracts, and automated checks.

The second system is not merely easier for an AI agent to work with. It is easier for everyone to work with.

The agent benefits because it has stronger boundaries and clearer feedback. Developers benefit because there is less implicit behaviour to remember. Code reviewers benefit because more correctness properties are mechanically enforced. Operations benefit because fewer invalid states can reach production.

AI simply makes the difference more visible because we are increasing the volume and speed of change.

If AI can produce changes faster than humans can safely verify them, then architectures that reduce the cost of verification become increasingly valuable.

Verification is a recurring cost

This is probably the most important economic point.

Suppose Architecture A costs 10% less to implement but requires significantly more effort to verify every time it changes.

Architecture B costs 10% more to implement but gives us strong structural guarantees that dramatically reduce the cost of verification.

If we only look at the initial project, Architecture A may look attractive.

But software is not a project that ends when the feature ships. It is a system that continues changing for years.

The implementation cost happens once. The verification cost happens again and again.

Every feature built on top of the system pays some of that verification cost. Every refactor pays it. Every migration pays it. Every AI-generated change pays it.

This creates a compounding effect.

A small amount of additional complexity that buys a strong, reusable verification guarantee can potentially pay for itself many times over.

This is why I think architecture discussions need to consider the lifetime economics of verification, rather than treating testing and verification as something that happens after architecture has already been decided.

Verification as a first-class quality attribute

We already talk about architecture in terms of qualities such as performance, scalability, reliability, security, availability, and maintainability.

I think we should increasingly ask another question:

How verifiable is this system?

Not simply:

"How easy is it to write tests?"

But:

"How cheaply and reliably can we establish that this system is behaving correctly?"

Those are different questions.

A system can have excellent test coverage and still be difficult to verify. It might have complicated state transitions, implicit dependencies, weak contracts, highly coupled components, or many possible invalid states.

Conversely, a system can be designed so that its structure itself provides useful guarantees.

That could mean strong types. It could mean database constraints. It could mean explicit state machines. It could mean narrow interfaces, deterministic components, schemas, transactional boundaries, idempotent operations, or other forms of architectural constraint.

The common property is that correctness becomes cheaper to establish.

This changes architecture trade-offs

I think this also changes how we should evaluate competing solutions.

Historically, we might ask:

Which solution is cheaper to build?
Which one is simpler?
Which one performs better?
Which one is easier to maintain?

Those questions still matter.

But I think we should add:

Which solution is cheaper to verify?
Which solution makes more invalid states impossible?
Which solution gives us stronger automated guarantees?
Which solution makes regressions easier to detect?
Which solution reduces the amount of behaviour that humans have to repeatedly reason about?
Which solution gives automated agents better boundaries and feedback?
Which solution reduces the blast radius of an incorrect change?

The interesting thing is that these questions can change the answer.

A solution that costs more to implement can be the economically better solution if it substantially reduces the recurring cost of verification.

And this doesn't mean the most constrained architecture always wins. Constraints have costs too. Excessive constraints can make a system unnecessarily rigid, difficult to evolve, or more complicated than the problem warrants.

The point is not to maximise constraints.

The point is to recognise that constraints have economic value when they reduce the cost of establishing correctness.

The question I now want to ask during architecture discussions

Coming back to that conversation at work, I still agree with the stakeholder's principle: the user should come first.

But I now think there is a missing dimension in how we reason about what is best for the user.

The question isn't:

"Which architecture is easiest for the AI?"

Nor is it simply:

"Which architecture is cheapest to implement?"

The question is closer to:

"Which architecture gives us the best overall outcome for the user, given the lifetime cost of building, operating, changing, and verifying the system?"

Sometimes the answer will be the simplest possible implementation.

Sometimes it will be the architecture that is easiest for an AI agent to modify.

Sometimes it will be an architecture with significantly more upfront complexity because that complexity gives us guarantees that would otherwise have to be repeatedly verified.

And sometimes the right answer will be something completely different.

The important thing is that verifiability is now part of the trade-off.

The architecture should help us prove that it is correct

AI-assisted development is often discussed in terms of how quickly we can produce software.

I think that is only half of the story.

If AI makes implementation dramatically cheaper, then our scarce resource increasingly becomes the ability to establish that what we produced is correct.

That means the architecture of a system is no longer just about making the desired behaviour possible. It is also about making incorrect behaviour difficult to express, easy to detect, and cheap to rule out.

The most interesting architectural decisions may therefore be the ones that turn verification from a recurring human activity into a property of the system itself.

Because the ultimate goal isn't to build software that AI can generate quickly.

It's to build software that we can continuously establish is correct.

And if we can design the architecture so that the system helps us do that, then verification stops being something we bolt onto the end of development.

It becomes part of the architecture.

Do We Still Need Code Reviews in the Age of Coding Agents?

Remo H. Jansen — Sun, 27 Sep 2026 08:25:14 +0000

For most of my career, code review has been a fairly simple idea.

One engineer writes some code, opens a pull request, and another engineer reviews it before it gets merged. The second engineer looks for bugs, questions design decisions, suggests improvements, and ultimately gives the team another level of confidence that the change is safe to ship.

It is such a normal part of software development that we rarely stop to question the assumptions behind it.

One of those assumptions is that another engineer wrote the code.

That assumption is becoming less reliable.

Today, a coding agent can implement an entire feature, run the tests, fix failures, refactor the implementation, and prepare a pull request. The engineer who opens that pull request may have spent most of their time reviewing and directing the agent rather than writing the code themselves.

This makes me wonder whether our traditional code review process still makes sense.

Not because I think code review is obsolete. I think the opposite is true: we need verification more than ever.

But I think we need to change how we think about code review.

What was code review actually for?

Before we decide what should happen to code review, it is worth asking what problem it was solving in the first place.

When one engineer writes a change and another engineer reviews it, we get a second set of eyes. The reviewer might catch a bug the author missed, notice that an abstraction is wrong, identify an edge case, question an architectural decision, or simply ask why something was implemented in a particular way.

There is also a social dimension to code review. It creates shared ownership and knowledge across a team. The code does not belong exclusively to the person who wrote it.

All of that remains valuable.

But not every part of the traditional process is equally valuable when the author is an AI agent.

I have started thinking about coding agents as something like flawed peers.

Imagine that you are a seasoned engineer and you have a very fast colleague who is capable of producing an enormous amount of code. They can work tirelessly, know an impressive amount about software development, and solve many problems very well. At the same time, they are inexperienced in your particular system and can occasionally make mistakes that seem obvious in hindsight.

You would not blindly merge their work.

You would review it.

You would give them clear constraints.

You would run automated checks.

You might ask another specialist to look at particularly important changes.

That sounds quite a lot like the agentic development workflow we are building now.

The engineer opening the PR is already a reviewer

This is where I think our traditional PR rules start to become interesting.

Suppose a team has historically required one human review for every pull request.

The traditional workflow looks something like this:

Engineer A writes the code → Engineer B reviews the code → merge.

Now consider an agentic workflow:

Agent writes the code → Engineer A reviews the code → Engineer A opens the PR → merge.

If Engineer A has genuinely reviewed the implementation, understands the change, and is willing to take ownership of it, what exactly are we gaining from requiring another human to perform a second generic review?

The PR being opened does not magically make the code more trustworthy.

The meaningful event happened before that: an engineer examined the work and decided that they were prepared to own it.

This is why I think we should be careful about treating approval counts as a proxy for quality.

If your previous process required one review, I think there is a reasonable argument that an agent-generated change which has been thoroughly reviewed by the engineer opening the PR may not need another generic human approval.

If your process required two human reviews, then perhaps the agent-generated change gets one human review by the owner and one additional human review after the PR is opened.

The exact policy will depend on the risk of the system and the kind of change being made.

The important part is the principle:

The number of approval buttons pressed is not the same thing as the amount of verification performed.

Trust the engineer, get ownership in return

There is also a cultural issue here.

If an experienced engineer reviews an agent-generated change, opens the PR, and says "I am happy to own this", I think we should take that statement seriously.

We often say that we want engineers to have ownership.

But ownership without trust is difficult to achieve.

If we tell engineers that they are responsible for the code but then require another person to validate every decision they make, we are creating a system where responsibility and authority do not quite match.

I would rather move toward a model where we say:

You reviewed it. You understand it. You own it.

In return, we should expect engineers to take that responsibility seriously.

This does not mean trusting every change equally. A production database migration, authentication change, or financial transaction workflow deserves a different verification strategy from a small UI change.

It means that our process should respond to risk and uncertainty, rather than blindly applying the same number of reviewers to every pull request.

The bigger problem is code volume

There is another problem that I think is even more important.

Coding agents make producing code dramatically cheaper.

That is fantastic.

It also creates a problem.

If generating code becomes cheap enough, the amount of code produced can increase much faster than the amount of human attention available to review it.

We cannot solve that problem simply by adding more reviewers.

If an agent produces ten times more code and our answer is to have humans manually inspect ten times more code, we have simply moved the bottleneck from writing software to verifying software.

Human attention is scarce.

So we need to become much more selective about where we spend it.

Don't review what you can make unrepresentable

This is where architecture becomes a much more important part of code review.

There are many classes of bugs where I would rather not depend on a reviewer noticing the problem.

I would rather design the system so that the incorrect implementation is difficult or impossible to represent.

Consider a database schema.

If the database requires a value to satisfy a particular constraint, we don't need to rely entirely on every application developer remembering to validate that constraint correctly. The database can enforce it.

Or consider a business workflow.

If a process can only move from Pending to Approved or Rejected, we can represent those states explicitly rather than allowing arbitrary strings and hoping that every piece of application code handles them correctly.

Finite state machines are a good example of this approach. The architecture itself describes which transitions are valid and which are not.

The same idea applies to types, contracts, API boundaries, validation, generated code, static analysis, automated tests, and many other forms of explicitness.

The goal is not to make developers more careful.

The goal is to make certain mistakes impossible.

That changes the economics of code review.

Instead of asking a human to inspect every line and think about every possible failure mode, we can move some classes of verification into the system itself.

The reviewer can then spend their limited cognitive capacity on the things that actually require human judgment.

Verification becomes a layered system

This suggests a different model for reviewing agent-generated code.

At the bottom, we have constraints and architecture that prevent entire classes of incorrect implementations.

Above that, we have automated verification: type checking, tests, static analysis, security scanners, contract tests, database constraints, CI checks, and whatever else is appropriate for the system.

Then we have the engineer who reviews the change and takes ownership.

And for changes where we want additional confidence, we can add another layer.

Adversarial agents.

What if the reviewers were agents too?

We don't necessarily have to choose between one human review and three human reviews.

We can introduce specialized agents whose job is not to approve the code, but to try to find reasons why we should not trust it.

Imagine an agent whose only responsibility is security.

It reviews the change and asks questions such as: "Can this input be manipulated? Did we introduce an authorization bypass? Are there new injection opportunities? Did this change expose something that should remain private?"

Another agent might specialize in performance.

It could look for expensive queries, unnecessary allocations, N+1 database access, contention, excessive network calls, or other performance problems.

We could have agents specializing in reliability, backwards compatibility, API design, testing, accessibility, architecture, or whatever concerns are particularly important to a given system.

The important part is that these agents have specialized responsibilities.

I don't want ten generic agents saying "LGTM."

I want adversarial agents trying to break my confidence in the implementation.

The security agent should be trying to find a security problem. The performance agent should be trying to find a performance problem. The architecture agent should be trying to find an architectural violation.

And, just like the coding agent, we should remember that these are flawed peers.

A security agent does not prove that our application is secure. A performance agent does not prove that the system is fast.

They provide another independent attempt to find problems.

That can still be extremely valuable.

More verification without a longer delivery cycle

This gives us an interesting alternative to the traditional approach.

Today, if a change is considered important, we might respond by adding more human reviewers.

That increases the amount of human attention required and can increase the time it takes to get the change through the development process.

In an agentic workflow, we have another option.

We can increase the amount of verification without necessarily increasing the number of humans involved.

For example:

Coding Agent
     |
     v
Engineer reviews and takes ownership
     |
     +------> Security Agent
     |
     +------> Performance Agent
     |
     +------> Architecture Agent
     |
     +------> Testing Agent
     |
     v
Automated verification
     |
     v
Merge

Not every change needs every agent.

A documentation change probably does not need a performance review. A database migration probably deserves more scrutiny than changing a button label.

The important thing is that verification can become composable.

We can assemble the verification pipeline according to the risk of the change.

Code review at scale

This leads me to a slightly different way of thinking about code review.

The answer to having more generated code is not necessarily to review more code.

It is to make less code require human review.

We can do that in several ways.

We can use architecture and constraints to make certain incorrect implementations unrepresentable.

We can use automation to verify things that machines are better at checking than humans.

We can use specialized adversarial agents to search for specific categories of problems.

And then we can use human engineers for the decisions where human judgment actually matters.

Does this solve the right problem?

Is this the right abstraction?

Does this fit the architecture?

Are the trade-offs appropriate?

Does this behavior make sense for the business?

Is this something we actually want to own and operate?

Those are difficult questions to reduce to a simple automated check.

Checking whether a database column is nullable is not.

Checking whether an API contract is backwards compatible is often automatable.

Checking whether every state transition is valid can be encoded into a state machine.

The more of these things we can move out of human cognition, the more effectively humans can review the things that remain.

The PR isn't dead

I don't think code reviews are going away.

I think the meaning of a code review is changing.

When humans wrote most of the code, the natural workflow was for another human to inspect the author's work.

When agents write most of the code, the engineer's role becomes less about being the person who typed the implementation and more about being the person who understands it, verifies it, and takes responsibility for it.

That doesn't mean we should blindly trust agents.

Quite the opposite.

It means we should build better verification systems around them.

Some verification should happen through architecture. Some through constraints. Some through automation. Some through specialized adversarial agents. And some through experienced engineers exercising judgment.

The goal shouldn't be to remove verification in order to move faster.

The goal should be to increase verification while reducing the amount of expensive human attention required for it.

Maybe the question for the agentic era isn't:

"How many engineers need to review this PR?"

Maybe it is:

"What is the cheapest reliable way to gain confidence in this change?"

Sometimes the answer will still be another human.

Sometimes it will be an agent.

Sometimes it will be a test, a type, a database constraint, or an architectural decision that makes the bug impossible.

And sometimes, if an experienced engineer has already reviewed the work and is willing to own it, perhaps the answer is simply to trust them.

Revisiting the Toyota Production System (TPS) in the Age of Coding Agents

Remo H. Jansen — Wed, 23 Sep 2026 23:36:43 +0000

Software has borrowed a surprising amount from manufacturing. Not from manufacturing in the sense of “let’s put programmers on an assembly line.” Thankfully, we’ve tried enough variations of that idea already.

I mean something more interesting.

Some of the most influential ideas in modern software development came from looking at how Toyota transformed manufacturing. Lean Software Development, Kanban, limiting work in progress, reducing waste, optimizing flow, continuous improvement, pull systems—many of these ideas have a lineage that leads back to manufacturing.

And at the center of that story is the Toyota Production System (TPS).

Toyota describes TPS around two pillars: Jidoka and Just-in-Time. Jidoka is roughly “automation with a human touch”: when an abnormality is detected, the process stops rather than continuing to produce defective products. Just-in-Time is about producing what is needed, when it is needed, and in the amount needed.

These ideas were developed for a world of physical products. But we are entering another period where software development is changing dramatically—and this time, the manufacturing analogy might be useful for a completely different reason.

I. The Software Bottleneck Is Moving

For most of the history of software development, producing software was expensive. Writing code took time. Understanding a codebase took time. Testing, debugging, and reviewing code all took time. Because developers are human, all of these activities were constrained by the number of humans available to perform them.

We built processes around this constraint:

IDEs to make writing code faster
Languages & Frameworks to make code easier to express
CI Systems to automate testing
Agile & Kanban to improve feedback loops and flow

Then coding agents arrived. Suddenly, the cost of producing code changed. An agent can read a codebase, modify dozens of files, run commands, inspect test results, make another change, and repeat the process without waiting for a human to type every line.

The production capacity of software development can increase dramatically. But if an agent can produce changes faster than we can establish that those changes are correct, we have created a new bottleneck: Verification.

TRADITIONAL SOFTWARE

┌──────────────┐
│ Human writes │
│    code      │
└──────┬───────┘
       │
       ▼
┌──────────────┐
│ Verification │
└──────────────┘

Production and verification are relatively balanced.

AI CODING AGENTS

┌──────────────────────────────┐
│                              │
│      CODE PRODUCTION         │
│                              │
│          ████████████        │
│          ████████████        │
│          ████████████        │
│                              │
└──────────────┬───────────────┘
               │
               ▼
        ┌──────────────┐
        │ VERIFICATION │
        │      █       │
        └──────────────┘

The factory produces faster than quality control can inspect.

That is a manufacturing problem, and manufacturing has spent a very long time thinking about it.

II. Looking at the Factory Floor Again

The answer is not to blindly copy Toyota. Software has no physical inventory, no physical assembly line, and no truck waiting for parts. But the underlying challenges are identical:

How do you prevent defects?
How do you detect problems as early as possible?
What should happen when an abnormality occurs?
How much unfinished work should be allowed to accumulate?
How do you improve the production process rather than merely inspecting its output?
How do you increase production speed without increasing the rate at which defects escape?

These are the exact questions that led to modern manufacturing quality practices.

III. Jidoka & The Software Andon Cord

Of all the Toyota concepts, Jidoka is the most interesting for AI coding agents. The basic idea is simple: if a machine detects an abnormality, it stops. The objective isn’t simply to detect defective products at the end of the line—it is to prevent the production of more defective products in the first place by building quality directly into the process.

A conventional automated system behaves like this:

Produce → Produce → Produce → Inspect → Discover Problem

Jidoka aims for something different:

Produce → Produce → Detect Abnormality → STOP

Imagine an agent working on a task. It modifies several files sequentially. Tests pass until an important invariant breaks. A naïve agent continues attempting fixes randomly:

Test failed. I’ll try another approach… and another… and another.

In doing so, it accumulates a massive amount of unverified state. A Jidoka-inspired agent behaves deliberately:

Agent Makes Change
       │
       ▼
  Verification
       │
       ▼
Abnormality Detected ──► STOP ──► Investigate ──► Fix ──► Verify ──► Continue

The agent should not merely be capable of stopping; it should be designed to stop.

The Andon Cord

Toyota uses an Andon signal to make abnormalities visible and draw immediate attention to the point of failure. While software already uses crude signals like failed CI builds or production alerts, AI agents allow us to make every invariant an Andon cord:

❌ Type safety violated
❌ API contract changed
❌ Security policy violated
❌ Migration is destructive
❌ Test coverage dropped
❌ Performance budget exceeded

The critical distinction is that the signal is not just a passive warning displayed on a dashboard; it is an active control mechanism. The agent loses permission to continue until the abnormality is resolved.

IV. Poka-yoke: Mistake-Proofing the Environment

Poka-yoke focuses on using fail-safe mechanisms to avoid simple mistakes entirely. We tend to frame software verification as:

“How can we detect whether the agent made a mistake?”

A far better question is:

“Can we make it impossible for the agent to make this particular mistake?”

If an agent needs to modify a database, giving it unrestricted access relies on post-hoc testing to catch dangerous operations. Designing a poka-yoke environment changes the interface:

┌───────┐      ┌──────────────┐      ┌──────────────┐      ┌──────────────────┐
│ Agent │ ───► │ Database Tool│ ───► │ Policy Layer │ ───► │ Allowed Operation│
└───────┘      └──────────────┘      └──────────────┘      └──────────────────┘

A destructive operation simply isn’t available through the interface. The agent cannot execute it because the capability does not exist in that context.

This applies equally to dependency installation, secret access, schema migrations, and permission changes.

Core Design Principle: Don’t teach the agent not to make a mistake when you can design the system so the mistake cannot be made.

V. Quality at the Source & WIP Limits for Uncertainty

An agent executing a dozen changes before running verification creates a massive search space when a failure inevitably occurs. Diagnosing which assumption failed becomes exponentially harder as unverified work accumulates.

AI coding agents require a concept similar to Work-in-Progress (WIP) limits. Instead of tracking open tickets, we should limit the amount of unverified change in the system.

    Make Small Change
            │
            ▼
         Verify
            │
            ▼
    Make Another Change
            │
            ▼
         Verify

Consider an agent operating with a Verification Debt budget:

Unchecked Invariant = +1
Unvalidated Dependency = +1
Unverified Integration = +1
Verification Debt = 0: Agent proceeds freely.
Verification Debt = 3: Agent must verify before taking further actions.
Verification Debt = 5: Agent is stopped completely.

VI. The Economics of Inspection

Not every verification step needs to carry the same execution cost. Cheap defects must be caught cheaply through a tiered inspection pipeline:

┌───────────────────────────────────────────┐
│              Syntax / Parsing             │  ◄── Fast / Cheap
├───────────────────────────────────────────┤
│               Type Checking               │
├───────────────────────────────────────────┤
│              Static Analysis              │
├───────────────────────────────────────────┤
│                Unit Tests                 │
├───────────────────────────────────────────┤
│               Contract Tests              │
├───────────────────────────────────────────┤
│             Integration Tests             │
├───────────────────────────────────────────┤
│              E2E / Behavioral             │
├───────────────────────────────────────────┤
│          Adversarial Verification         │  ◄── Slow / Expensive
└───────────────────────────────────────────┘

If an agent references a non-existent function, it should not require a 45-minute integration suite to catch it. In a high-throughput environment where an agent generates dozens of candidate changes an hour, optimizing the verification loop itself is critical to preventing systemic bottlenecks.

VII. Process Control and Root-Cause Analysis

Statistical Process Control

In traditional manufacturing, quality control monitors the process, not just individual outputs. If an agent’s defect rate suddenly spikes, focusing only on single test failures misses system-level shifts (e.g., context window truncation, degraded model weights, altered tool behavior, or stale API specs).

Tracking process-level metrics provides visibility into the health of the system:

Metric	Target Baseline	Out-of-Control State
Compile Failures / 1,000 Changes	0.7%	4.8%
Test Failures / 1,000 Changes	3.1%	8.2%
Reverted Changes	0.9%	5.7%
Security Findings	0.03%	0.11%

When metrics breach baseline thresholds, the relevant question isn’t “Is this code change bad?” but “Has the production process itself gone out of control?”

Human Review Shifts Upward

When an engineer oversees dozens of autonomous agents, manual line-by-line code review becomes impossible. The human role shifts from reviewing individual diffs to evaluating the integrity of the generation process: verification coverage, failure patterns, architectural invariants, and root causes.

Kaizen (Continuous Improvement)

When a failure occurs, the agentic loop should produce process knowledge rather than simply retrying:

Test Fails ──► Containment ──► Identify Abnormality ──► Root-Cause Hypothesis ──► Fix & Regression Protection

Every failure should permanently update the system constraints:

Agent Task Fails (Stale API Docs)
              │
              ▼
    Add Contract Test
              │
              ▼
   Update Agent Constraint
              │
              ▼
  Improved Production System

VIII. End-to-End Traceability

When a defect manifests in production months after deployment, tracing its origin in an agent-generated codebase requires a complete provenance chain:

Requirement → Agent Session → Model/Version → Context & Tools → Verification Logs → Human Approval → Commit

Understanding defect origin shifts our diagnosis from “Who wrote this code?” to “What production process generated this code, and where did that process fail?”

IX. Conclusion: The Quality System Is the Deliverable

The conversation around AI coding agents focuses heavily on the agent itself: which model, which benchmark, which context window, which reasoning capabilities.

These factors matter, but the system surrounding the agent will ultimately matter more. A mediocre agent operating inside an exceptional verification environment will consistently outperform a brilliant agent operating inside a weak one.

The future of autonomous software engineering is less about building an AI that never makes mistakes, and more about building a production system in which mistakes are cheap, contained, visible, and quickly corrected.

The first wave of Lean software development optimized for flow, reduced batch sizes, and shorter feedback loops. The next evolution will focus on verification. As production costs drop toward zero, quality engineering becomes the primary constraint.

The most important question in modern software development is no longer “How do we build software faster?”

It is: “How do we build a system that produces software extremely fast without letting defects escape just as fast?”

DEV Community: Remo H. Jansen

Your Platform Engineering Investment Is Not Paying Off. Here's Why.

1. The last 10% matters more than you think

2. Developers don't fall in love with roadmaps. They fall in love with experiences.

3. The hidden cost of making developers dependent on the platform team

4. Why can't developers help fix the platform?

5. AI makes this opportunity much more compelling

6. Your tooling choices can either enable or undermine collaboration

7. Platform engineering needs a product mindset

8. Measure value, not just activity

9. What should organisations do differently?

Conclusion: The return on platform engineering is earned after delivery

From Never Getting Selected to a 1-in-3 Success Rate: Did AI Help Me Write Better CFPs, or Is AI Selecting Them?

Halo is the new Doom and most software will soon be effectively open source (by force or by choice)

Halo everywhere — including 100+ player co-op on PC

Demon's Souls Remake: from ~1 FPS to 60+ (and nearly 120) in roughly a month

REA — Reverse Engineer Anything

The Adobe and Office “Craft” ports — imperfect, but the signal is loud

What these stories share

Where is this going?

SDLC != ADLC != ASDLC

The Three Terms, Cleanly Separated

Why the Mix-Up Happens

Why the Distinction Actually Matters

Quick Mental Model

The Agentic Manifesto: What Should the Agile Manifesto Look Like in the Age of AI?

Why revisit the Agile Manifesto?

From Agile to Agentic

1. From producing software to verifying outcomes

2. From implicit assumptions to explicit intent

3. From supervising implementation to designing autonomous systems

4. From optimising implementation to optimising the whole system

The Agentic Manifesto

Principles behind the Agentic Manifesto

What does this mean for the future of software development?

What do you think?

Mutation Testing with TypeScript: How to Verify Your Tests in the Age of AI Coding Agents

1. Let's start with a simple problem

2. What is mutation testing?

3. Code coverage isn't the same as test effectiveness

4. Let's set up StrykerJS

Initialize StrykerJS

Run your first mutation test

5. Understand the mutation report

What about the mutation score?

6. Improve the tests and run mutation testing again

7. From local experiments to pull request verification

Should mutation testing block every PR?

8. Let the AI agent investigate surviving mutants

9. Mutation testing is one layer in a Swiss cheese model

10. Why this matters when code generation scales

Conclusion

Further reading

From Building Products to Building Factories: The Industrial Revolution of Software Engineering Is Here

What changed during the Industrial Revolution?

The software industry's industrial revolution

1. Hand tools → Machines → Coding agents

2. Small workshops → Factories → Agentic software factories

3. Human and animal power → Steam engines → Compute and datacentres

4. Canals and railways → Orchestration and automated workflows

5. Craft production → Standardisation and repeatable processes

6. Manual production → Factory operation and management

Engineers will build the factories that build software

The engineer becomes a factory architect and operator

Why the Toyota Production System matters

The new bottleneck is not necessarily coding

From engineering capacity to production capacity

The factory is not the datacentre

The Industrial Revolution is about the system, not the machine

Building Agentic Engineering Resilience: Using VS Code Copilot Automations + Datadog MCP for Recurrent Error Detection and Fixing

1. Automations (Preview) in VS Code GitHub Copilot

How to enable and create one

2. Enabling the Datadog MCP Server in VS Code

Critical warning: the 128-tool limit

3. A Verification Layer for Recurrent Errors

Skills (slash commands / reusable prompts) you can develop

Putting it together in an Automation

Why this matters

From T-Shirt Sizes to Token Quotes?

Estimation never really worked