When agents write most of the code, the scarce work moves downstream. Generation gets cheap, and the bottleneck moves to review and merge. In my reading, onboarding and choosing what is worth building get more expensive too. Five episodes on Chain of Thought each point at one change I would make in that world.
What should you review when you stop reading every line?
Anush Elangovan, Corporate VP of AI Software at AMD, lays this out on 250,000 Lines of Code a Week. He runs 10 to 12 Claude Code agents across three machines and burns 6.5 billion tokens a week. The claim I keep from that episode is that testing is the new code review. The chapter list frames the habit as 80% tests first. He argues that normal SDLC is dead in an agentic world, and that software comes down to tokens. In his account, speed is the only durable moat. The same conversation covers a 25-year-old Slurm replacement, rewritten as Spur in Rust in a single night, without opening the editor. An earlier chapter asks when AI-generated code becomes a liability. He describes agents as sneaky and dumb, and he runs them with dangerously-skip-permissions. I would leave that flag as his risk setting. I would start by requiring tests before an agent is allowed to widen the change.
# Sketch: a local check before merging agent output.
# Not a workflow used by any guest or company on these episodes.
def merge_allowed(change: dict) -> bool:
if not change.get("tests_written_first"):
return False
if not change.get("tests_passed"):
return False
return True
How do you merge when agents open pull requests in the background?
Greg Foster, co-founder and CTO of Graphite, describes the review bottleneck on After Code Gen. He argues that the bottleneck has moved from writing code to reviewing it, once agents open pull requests in the background faster than humans can merge them. He outlines three waves of AI that lead to autonomous agents spawning those pull requests. He argues that as AI automates the inner loop of writing code, the outer loop of reviewing, merging, and deploying comes under pressure. The tactic I would copy is stacking. Split a large change into units small enough for a human or an agent to review. He also identifies a hiring gap. Experienced engineers with strong architectural context become lethal with AI tools. If you hold that context, spend it on how changes are split and how they land.
How do you onboard an agent that will own most of a day?
Sterling Chin, Applied AI Engineer and Senior Developer Advocate at Postman, treats staffing as the method on I Built an AI Coworker That Runs 90% of My Day. He stopped thinking of AI as a tool and started treating it like a junior employee. He onboarded it with context, corrected its mistakes, and gave it writing rules. Forty days later, MARVIN, built on Claude Code, was handling 90% of his workday. The episode is built around live demos. MARVIN bookends the day from first login to the end, using personality, sub-agents, and writing rules. The demos run from meeting transcripts to Jira tickets, drafted posts, and calendar management. Show notes call MARVIN a full-time AI chief of staff. Sterling has onboarded 12 or more colleagues at Postman, including non-technical knowledge workers, and he argues that DIY assistants outperform big tech alternatives. I would start with one assistant, written rules, and a correction habit. At a fixed checkpoint, count how much of the day it actually runs. Forty days and 90% are his reported outcome.
What should a coding model be scored on?
Jason Warner and Eiso Kant, CEO and CTO of Poolside, make the training case on First Code, Then AGI. They explain why Poolside trains for code specifically instead of general capability, and what reinforcement learning against real execution feedback buys a model. They believe synthetic data will help expand the surface area of software by 1000x. Their focus includes high-consequence systems. On the human side, they argue that agency, taste, and judgment become central for engineers working with AI coworkers. The buying decision I would make is narrow. Prefer a coding agent whose feedback loop includes real execution, and keep a person with taste on whether the passing behavior is the one you wanted.
Where should you point agents if you want a measurable payback?
Rachelle Palmer, Director of Product Management at MongoDB, covers the least glamorous case on Using AI to Modernize Your Legacy Applications. The collection page describes pointing AI at legacy codebases as one of the few coding-agent uses with an obvious payback measured in years off a timeline. Her forward-deployed AI engineering team works on complex, outdated codebases. She details LLMs automating documentation improvements, test generation, and business logic conversion. In her account, modernization timelines drop from years to months, and work that once demanded teams of dozens can be done by a small team. I would start on a legacy system already stuck on a multi-year plan. Constrain the agent to documentation, tests, or a bounded logic conversion, then compare the calendar you had with the calendar you have.
Checklist
- Make testing the review gate. Anush Elangovan's episode frames this as tests first, with agents he describes as sneaky and dumb.
- Stack changes into reviewable units, the tactic Greg Foster gives once background pull requests are the bottleneck.
- Onboard one assistant like a junior employee, with context, corrections, and writing rules, then check results on a set day, as Sterling Chin reported after forty days.
- Prefer coding models trained with reinforcement learning on real execution feedback, the bet Jason Warner and Eiso Kant describe, and keep taste with a human.
- Point the first large agent effort at legacy modernization, where Rachelle Palmer describes timelines moving from years to months.
More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.
Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.
Drafted with AI assistance from the episode transcripts.
Top comments (0)