Picture a solid mid-level engineer joining your team. Nobody gives them access to the issue tracker. Nobody shows them the knowledge base. Architecture decisions were explained once, during onboarding. Occasionally they get asked to look something up, sometimes they get handed a small feature with no sense of the bigger picture, and then they get blamed for not understanding the task or the way things work here.
That is roughly what adopting AI in development looks like today. What follows is about the rakes teams step on when they bring agentic development into their process, regardless of team or company size, and about what I have worked out and borrowed from others over years of pair programming with computers.
The same mistakes show up in startups, in venture studios, at non-technical solo founders and, predictably, deep inside enterprise. Some of them already call themselves AI-native, while every single employee works with a single agent in their own private way, and knowledge gets exchanged only when someone takes personal initiative in a private conversation.
Developers get a Claude Code or Codex license and an invitation to figure it out themselves. Management heard that agents speed up development, the tool has been handed out, the box is ticked. Delivery speed somehow stayed exactly where it was.
It stayed there because exactly one node got automated, and the system around that node did not change. Code is now generated at inference speed, and the sprint is planned the way it always was.
Peter Steinberger, in Shipping at Inference-Speed, describes a workflow where his throughput is limited by inference time and by genuinely hard engineering decisions. He runs several tasks in parallel, maintains documentation per subsystem, lets agents execute commands and verify their own results.
He is an excellent example of what an experienced solo developer achieves after rebuilding his entire personal environment around agents. The same piece carries an important caveat: some of those practices will not survive contact with a large team.
For one developer with a well-defined task, inference speed really does become the main constraint. For a whole team, the constraint is the process that generates those tasks in the first place.
Back in the pre-agent era, when I wrote about using neural networks to speed up my own work, the question I heard most often was:
But what about highload?
The context was always whether these approaches hold up in large systems, and I could not see how the size of a system prevents you from delegating specific tasks to a model. Then it clicked: people were imagining exactly one usage scenario. Stuff the entire codebase into the context, ask the model to behave like a senior engineer, wait for everything to get designed and start working by itself through some kind of magic, then sigh with disappointment.
Neither a model nor a human works that way.
A person who knows a project well keeps routes in their head. They know where things live, where the contract is, who to ask when the contract stays silent. An agent needs precisely the same thing.
The route looks roughly like this:
intent or task
↓
acceptance criteria and risk boundaries
↓
routing to the right context
↓
plan for any non-trivial change
↓
isolated implementation
↓
mandatory local checks
↓
independent review and CI
↓
staging and behavior verification
↓
controlled delivery
↓
telemetry, conclusions, documentation update
Removing humans from every node is not the goal. Every segment needs an explicit input, output, owner and transition condition. Where risk is low, the transition is automatic. Where the cost of a mistake is high, a human confirms. The agent executes the part of the process it is allowed to execute and carries a verifiable result to the next gate.
Take a concrete example. A team is adding idempotency to payment creation. The task records a checkable promise: two requests with the same key create one operation. The root instruction routes the agent to the API contract, the payments ADR and the migration rules. The feature plan lists affected components, the state after a partial failure and the rollback path. The agent changes code in an isolated environment.
Then it runs the happy path together with a repeated request, parallel requests, a crash after the database write, and a re-run of the migration. A second agent, or a human, receives the original promise, the diff and the check results. CI re-executes the mandatory profile. On staging the scenario runs against synthetic data. Once that passes, a human accepts the result and approves delivery of the built artifact to production.
The model writes code here. Reliability comes from the process around it: criteria defined up front, the right context, isolated implementation, independent review, repeated checks, and a human decision about delivery.
Here is what that route is built from.
One entry point beats an encyclopedia in a single prompt
The project root needs a short file the agent reads before doing anything. In Codex that is AGENTS.md.
Anthropic documentation states that Claude does not read AGENTS.md and requires a dedicated CLAUDE.md. In my experience it reads it just fine when no alternative file exists. On top of that, AGENTS.md can be named as an explicit entry point in the initial instruction when needed, and maintaining a duplicate CLAUDE.md with identical content buys nothing.
The root file exists for routing. Inside, it is enough to describe the purpose of the project and its boundaries, the commands for running and basic verification, a map of the documentation, rules for picking a test profile, actions that require confirmation, and a definition of done.
It can look something like this:
# Working agreements
- Read `docs/contracts/` and `docs/testing/api.md` before changing the API.
- Follow `docs/workflows/migrations.md` for migrations.
- Start non-trivial tasks with a plan in `docs/plans/active/`.
- Update user-facing docs after any behavior change.
- A task is done when the check commands and their output are attached.
- Prepare report and instructions for an operator.
The rule is selected by the type of change. A minor CSS tweak gets away with a linter and component tests. A payment contract drags the full testing profile behind it.
This helps humans too. A new hire opens the same repository, walks the same routes, and works out faster how things are done here. Good documentation for an agent almost always turns out to be good documentation for the team. Implicit agreements hurt everyone equally.
Documentation lives in the repository
Documentation for agents is a shared, versioned, maintained repository of knowledge about how the product gets built. If agentic documentation in your team is every developer's private business, standardized development and quality control are simply not on the table.
The funniest place to find this missing is a development studio that calls itself AI-first. Product delivery is running as a conveyor belt, while process and stack standardization was done, at best, on advice from ChatGPT. It is fashionable right now to mock non-technical people who finally got the ability to build their ideas without a computer science degree, but studios like these contribute at least as much to mass slop generation. Probably more, because churning out barely maintainable projects with several pairs of hands goes much faster.
Once agentic documentation is standardized for teams, the processes inside start to shift. Knowledge stops surfacing in random links and internal workshops. Code and documentation changes flow through one review stream. A new developer receives current docs along with project access. The agent sees status and scope right next to the code, and behaves identically on every single workstation.
A minimal documentation structure can look like this:
AGENTS.md
docs/
architecture/ # subsystems and data flows
contracts/ # APIs, schemas, external commitments
decisions/ # ADRs and the reasoning behind them
plans/
active/ # in-flight changes
completed/ # finished plans with outcomes
research/ # verified research notes
testing/ # test profiles and selection criteria
workflows/ # repeatable processes
operations/ # runbooks, diagnostics, rollback
Directory names can differ. What matters is what the documents are for and how clearly they link to each other.
In my own projects and in client work, alongside general documentation, I keep several document types specific to working with an agent.
The master plan
The master plan fixes the direction of the project: the goal, the boundaries, architectural principles, major stages and phases, external dependencies, key risks, and what is deliberately not being done.
This document usually gets created in a new project that starts from a blank page and research. Once the necessary data is collected, the boundaries are clear and the final shape has formed, at least at the current level of understanding, the master plan gathers all of it in one place and gives you a base for producing the next documents quickly.
Researching the main direction of a project is best done inside one session, and the master plan is best written inside that same session. When the research does not fit the session context, or starts diluting it, save intermediate results as separate documents. Later they work as bouillon cubes of context for brewing the big picture in the master plan.
The feature plan
The feature plan describes a specific change: current and desired behavior, affected components, invariants, migration, failure scenarios and readiness criteria.
Such a plan is usually a description of a process that should lead to a particular result. It is fine for this document to be large and to cover more than one feature. When it gets big, it splits into phases, each with its own entry conditions and its own completion and acceptance criteria. All of them push in one direction: build new end functionality, run a large refactor, or stabilize part of the system.
The ADR
The Architecture Decision Record stores the reason behind a decision, the options considered and the consequences. An ADR exists to explain and record a decision made on the fly, for example when new data appeared. That decision may differ from what a specific feature plan says, or deviate from the general line of the master plan. The ADR records why, under what conditions this was decided, and why it matters. It is simultaneously a hard record of knowledge and one of the instruments of flexibility in the process.
A document needs a lifecycle
A docs/ folder standardizes nothing by itself, and without a process it quickly turns into a dump of markdown files.
Every important document needs at minimum a scope, a status, an owner, a last-reviewed date, and a condition that triggers a review. Four statuses are enough: draft, active, superseded, archived.
Updating documentation belongs in the definition of done. If a change touched a public contract, a mandatory command, an architectural invariant or an operational process, the task is not finished until the corresponding document is updated.
Documenting everything, on the other hand, is unnecessary and harmful. Directory structure, the dependency list and function signatures are things the agent reads out of the repository by itself. What needs documenting is what the code cannot tell you:
- intent;
- constraints;
- reasons behind decisions;
- dangerous exceptions;
- acceptance criteria;
- known operating modes and when they apply.
Conflicts between sources get resolved explicitly. Tests show recorded expected behavior, code shows the actual implementation, the ADR explains architectural intent, an active plan describes a change that is not finished yet. When they diverge, the agent stops, records the conflict and escalates it to the owner. A machine must not have the option to pick whichever version of the truth is most convenient.
Mechanical checks move into CI: broken links, missing mandatory fields, unknown owners, a completed plan with no final status. Whether a document is still true in substance is confirmed by the same owner who is responsible for the corresponding system behavior.
Where to put knowledge so that migration does not cost you weeks
Not in skills. There is a lot of mysticism around agent skills right now, when in essence a skill packages a repeatable process: instructions, supporting material, sometimes scripts. OpenAI describes skills as a format for reusable workflows with progressive context loading.
Skills work well when a task genuinely repeats: fixing a particular class of pipeline failure, preparing a migration from an established template, assembling release notes. Putting the entire memory of a company into them makes no sense.
First, a given agentic system can change the format, the loading rules or the behavior after any update. Second, a vendor can become unavailable to a specific team, region or account.
Here an author describes the full pain of migrating off skills from Claude Code to Codex, after Anthropic started enforcing regional account blocks more carefully. Translated from Russian:
The main problem was the skills. Over the past few months I had grown a very deep layer of skills for internal systems, some of them duplicating each other.
Reworking the skills took two and a half days. The difficulty was that for some internal systems no ideal skill exists yet. For our version control system, for example, there are three skills with different strengths and weaknesses. And three different authors.
over time the whole zoo of skills turns into a landfill and you have to clean up.
That last quote resonates with me most, because documenting skills produces exactly that feeling. Documenting approaches seems more correct than documenting specific abilities.
A skill instruction that one model executed well is under no obligation to be executed as well by another. Commercial models get updated, older versions eventually go away, and what worked reliably with GPT-5.5 starts misfiring in 5.6, or gets ignored entirely.
So I separate the layers of documentation:
- project knowledge, invariants and mandatory policies live next to the code;
- a repeatable workflow can be packaged as a skill;
- critical and deterministic actions are implemented as a script or a check;
- the adapter for a specific agent only wires up the shared source and its tools.
When migrating from Claude Code to Codex means moving hundreds of rules by hand, that is not a nice opportunity to clean up the old stuff. It is a sign that the knowledge grew into one runtime too tightly.
A separate skill needs testing too. The tasks it should work on, the tasks it should not, expected inputs and outputs, forbidden actions. Before switching a model or a version of the agent harness, those checks get run again. A newer, smarter model can just as easily make things worse.
The model is chosen by the cost of a mistake
When starting a project in an unfamiliar domain, it makes sense to hand research and design assistance to a strong model, and to give well-specified implementation to cheaper and faster ones.
The smart ones write documentation, the dumb ones do implementation.
Deciding who gets what should account for:
- the cost of a mistake;
- the volume and heterogeneity of the context;
- the reversibility of the change;
- the quality of automated verification;
- the complexity of the architectural decision.
A simple model implements a routine adapter against a precise contract perfectly well. A strong one is needed for a tangled refactor or a review worth reading.
Encapsulate the environment
Once an agent can run commands, install dependencies and change files, a reproducible environment has to become part of the contract.
Many teams do not standardize this at all. The project runs on the host machine, outside any container with a declared configuration, and breaks at the build stage because local library versions do not match. Now imagine a developer handing that problem to an agent with no rails, the agent adapts the project to that developer's local library version, and the fix rides to review together with the feature. Everyone's joy needs no description: "time gets spent undoing nonsense, and the culprit is, of course, the stupid agent that does not get software development."
Containers are necessary because they provide isolation, portability and the reproducible result that process standardization depends on.
- the environment comes up through a documented command or set of commands;
- dependency versions are pinned;
- external services have reproducible test doubles;
- state is created and cleaned up safely;
- available directories and network destinations are restricted;
- the same set of checks runs locally and in CI;
- a failed run does not affect the developer's machine.
In some places this means compose with a database, a queue and mounted source code so changes are picked up without a rebuild. In others a lightweight devcontainer is enough. In others you need a separate remote sandbox.
A container is not a magic pill on its own. Badly drawn security boundaries, mounted directories, privileged mode, a Docker socket or excess network access, open the host or the internal infrastructure back up to the agent, and that needs watching too.
Tests verify the promise of the task
An agent that wrote code and reported done has not proven the task is actually complete. The proof is the checks passed against the compliance matrix and the tests written back at the phase planning stage.
In my own practice and documentation I use three levels of check strictness:
- Strict verification with a full test run, before merge or delivery.
- Local and integration checks of the feature itself and the immediately surrounding area, without a full run, to confirm the implementation works and does not break directly affected parts of the system.
- Full YOLO, the vibe-coding mode, acceptable in small projects, experiments and early development, when there is still too little code to put everything on rails and you can just push feature after feature, throwing away and rewriting entire layers of the project.
Even a good, reliable run of tests and matrices is not a hundred percent guarantee that (a) everything is done, and (b) everything is done correctly. For work to count as ready for human acceptance, a second opinion is required.
Cross-checking
A five-step cross-check is roughly what works:
- The implementer agent, a simpler and cheaper model, receives the task, the context, and permission to change a defined limited area.
- The reviewer receives the original requirements, the code and the test results. It does not see the first agent's report.
- It performs what is essentially a review, hunting for specific classes of problem: missed requirements, hacks, security errors, missing tests, documents out of sync.
- Findings are shaped into a fix plan.
- The fix plan goes back to the implementer.
Sometimes one more check-and-fix iteration is needed and the task is done. Sometimes even after a second fix plan the weak model cannot get there. In that case it is reasonable to hand the finishing touches to the same smart model that did the review. There is not much left to do, and the token spend will be nowhere near what a full implementation by that smart model would have cost. What you get is a literal senior-to-junior relationship.
There is also a running joke that mentioning in Claude Code's instructions that Codex will be checking the result raises the default quality of self-checks, and therefore of the output. How well that actually works is hard to say. The principle of a second opinion, though, is one of the most useful practices there is, especially for resource-heavy tasks.
For risky changes, cross-planning helps: one smart model drafts the plan, another tries to break it before implementation starts. The conclusions are drawn by a human, and the decision about the final shape of the plan stays with the human.
When AI can be allowed into production
The short answer: never. Direct access with the right to change a running system is out of the question.
Every principle you need was invented before AI. Least privilege, separation of duties, audit, reproducible artifacts and rollback have not gone anywhere.
The practical boundaries are these:
- the agent does not get universal production keys;
- the agent does not SSH into a production host;
- the agent does not bypass CI/CD and does not deliver a locally built artifact;
- the agent does not change infrastructure, secrets or permissions on its own decision, not even on dev or staging.
Letting the agent see production data is fine. It can help with diagnostics through monitoring infrastructure that already exists. Building magical monitoring on top of an agent that roams production freely and decides for itself what is going on there is a poor idea.
An agent can prepare a change, open a pull request, assemble a migration plan, even trigger a pipeline, as long as triggering it grants no way around the gates to production. The decision to apply a high-risk change stays with a human, in a separate controlled loop.
The agent can have its fun in other environments. Staging accepts a release candidate that has already been verified, and there it can be seen almost live. Staging must not become a dumping ground for any output an agent produces. That is what dev is for.
How to tell whether the automation actually works
Lines generated, tokens spent, subtasks closed by an agent, pull requests opened, none of these indicate value on their own. A fast generator grows the review queue along with the volume of garbage in it.
Ghostty is the example. At some point the review queue swelled so badly that Mitchell Hashimoto rewrote the contribution rules. In his words, a bad pull request used to cost its author time and effort, and agents removed that natural limiter. The result was roughly ten times more bad issues and PRs. Random AI-generated pull requests now get closed on sight, and people who keep dumping slop on the project get banned for good.
Automating a single node of development, by handing out access to code generation tools piecemeal, speeds up only the passage of tasks through that node, and increases its idle time. The rate at which tasks are generated and how they move the project and the work forward is what metrics need to cover. How long it takes from a finished task to production, how long it waits at individual gates, how often it comes back for rework, and how many changes need a fix or a rollback after delivery.
The whole system around development has to move at the same tempo. If your tasks get planned once a week or once every two weeks, it does not matter how fast they get executed. The planning procedure itself caps how much you can deliver.
Task quality matters too. If everything is faster at every node but the project still stalls on the roadmap, you may have buried yourself in optimizations, or something in the pipeline broke, and all that speed goes into plugging leaks instead of moving forward.
If developers started generating code faster, tasks fly across the board from column to column, and delivery time did not change, you have found the bottleneck of the process. That is the thing to automate, or to investigate.
Where to start
A universal agent orchestration platform is not needed at stage one. It is enough to take one repeatable type of change and walk it through the full route.
Start with a few questions:
- Have we documented the critical instructions and policies?
- Is that documentation available to everyone in a single repository?
- Does that documentation work no matter which agent is reading it?
Then take one small subproject, or carve one out, or pick the class of tasks that looks the most optimizable, and make sure the answer to every question is a confident yes.
If something exists only as a request in a prompt, it is not automated. A process known to one developer is not standardized. If something goes wrong and the documentation has to be ported to the specifics of another agent, there is no maintainable documentation in the project yet.
Claude Code, Codex and other agents really do speed up development radically. At what cost depends on how responsibly you adopt them. A purchased license changes exactly one stage of the process, and the production system has to be rebuilt by hand.
Automation begins where every task gets a reproducible and verifiable route. The agent understands intent and finds exactly the context it needs. It works in a restricted environment, proves the result with tests and passes independent gates. It cannot expand its own permissions. The knowledge it produced returns to the shared repository, so the next developer and the next agent start from a non-zero mark.
That is when a team's ability to safely ship changes speeds up. The output of an individual developer is only one of the indicators.
Until then, whatever agents you hand your team, you have automated code generation, not development.
Top comments (1)
The tool-amplifier framing is right. My worry: the rationale does not survive a rebase and squash. Do you keep an authoring note per change, or let the diff carry it?