If your coding agent can draft, review, test, push, and merge, the question that matters is how much it costs when a step is wrong. How smart the model looks on a sample task comes second. Give as much autonomy as the risk of the task allows, and no more. Climb one rung at a time and decide at each step whether a human still signs off.
What are the five rungs on a coding copilot?
Aishwarya Srinivasan (Fireworks AI) used a coding copilot as her example on Chain of Thought episode 42, asking how far to let it go. The five activities are hers; arranging them as a ladder with risk notes is the AI, decoded page's reading, which I follow here.
- Write the code: lowest stakes. The agent drafts; you still own review, tests, and shipping. Usually a safe handoff when a human reviews the result.
- Review the code: the agent checks work as well as producing it. Still useful and still relatively low risk because a human is downstream.
- Write the tests: the agent decides what “correct” looks like. A bit more trust, because bad tests can hide real bugs.
- Push the code: the agent takes an action with consequences outside its sandbox. Depending on risk, this is a natural place for a human checkpoint.
- Merge to main: full autonomy on the shared codebase with no one in between. If you automate the whole chain and something breaks, you need to trace back through every step to find where and why. Without logging and checkpoints along the way, that becomes extremely hard.
Do we let it write the code for me? Should I let it also review the code for me? Should I also let it write unit tests for me? Should I also let it push the code and like merge the code for me?
Aishwarya Srinivasan, Fireworks AI, on Chain of Thought ep 42
I treat these as five separate permissions.
How do risk and compliance move the checkpoint?
There is no single right answer for every team. Srinivasan ties the ladder directly to risk and scrutiny.
The level of autonomy versus the level of human intervention that you want to have in any particular system is very subjective to what's the level of risk that you have, what is the level of scrutiny or compliance that you're building that under.
Aishwarya Srinivasan, Fireworks AI, on Chain of Thought ep 42
High-risk or regulated work earns a human in the loop. Low-risk work may not. Model risk management names the broader practice: any model in a consequential decision is a source of risk (wrong answers, bias, misuse outside its valid range, drift as the world changes). Structured validation before deploy, monitoring in production, documented limitations, and clear ownership are how you defend the choice when it matters. You do not need a regulatory mandate for that mindset to apply to merge rights on main.
Put checkpoints where the risk is.
What evidence do you need before you move up one rung?
Srinivasan argues you should quantify each step the system takes: at every checkpoint, is the step doing the right thing on your data? Evaluations at those checkpoints beat a vibe check, such as asking a familiar coding question, seeing the model do well, and calling it tested. She also stresses observability: metrics and traces you can inspect on a dashboard.
Before I allow the next rung, I want a bar I can explain to a teammate:
- Write: humans catch serious issues in review; each task has a trace (prompt, tools, diff).
- Review: agent review flags align with human judgment on a sampled set of PRs.
- Tests: injected known bugs fail; the suite is not structurally unable to fail.
- Push: narrow credentials, logged push events tied to trace IDs, rollback tested.
- Merge: all of the above plus a written release rule (who can approve, what evidence counts).
At push and merge, instructions in the prompt are not a control by themselves. AI guardrails are the boundary around the system that catches bad behavior the model might produce: block unsafe or out-of-scope input, scope permissions and require approval for high-stakes tool calls, filter outputs before they ship. The important property is enforcement outside the model: validation code, permission systems, approval gates. Another paragraph in the system message does not count.
Like if there is a point where I have automated the entire pipeline, right from where it understands what I wanted to do, it codes it out, it writes unit tests for me, it does a code review, It pushes and merges the code into the main branch. If something breaks, how exactly do I go back and see where it broke and why it broke? And can I really reverse each and every step that happened along the way? And that is really important when you're trying to think about fallback options.
Aishwarya Srinivasan, Fireworks AI, on Chain of Thought ep 42
If you cannot answer that quote for your pipeline, stay off rungs 4 and 5.
How can you rehearse the merge decision without shipping?
The merge rung is a judgment call under uncertainty. Would you merge this? is a short lab: inspect two pieces of evidence, choose a release decision, and compare the costs. I would use it to calibrate how your team weighs traces, tests, and gut feel before you grant merge to an agent in a real repo.
For longer-form context on agents as software (architecture, evals, production workflows), the best episodes for engineers collection on Chain of Thought is where I send people who want the systems view Srinivasan keeps returning to: build the agent as software with orchestration, memory, tools, and quantified checkpoints.
How would you encode the ladder?
A sketch; your own policy will differ.
// Sketch: map agent actions to ladder rungs
const RUNG: Record<string, number> = {
write: 1,
review: 2,
tests: 3,
push: 4,
merge: 5,
};
function allowed(
action: keyof typeof RUNG,
cap: number,
evidence: {
traceId?: string;
humanOk?: boolean;
testsCatchBug?: boolean;
}
): boolean {
if (RUNG[action] > cap) return false;
if (action === "tests" && !evidence.testsCatchBug) return false;
if ((action === "push" || action === "merge") && !evidence.traceId)
return false;
if ((action === "push" || action === "merge") && !evidence.humanOk)
return false;
return true;
}
Set cap from your risk tier. Promote the cap only when the evidence row for that rung is boringly repeatable.
Checklist
- Split permissions across write, review, tests, push, and merge instead of one autonomy knob.
- Set the highest allowed rung from risk and compliance.
- Require traces plus evals at each checkpoint before you promote the cap.
- Use code-enforced guardrails and human approval on push and merge when stakes are high.
- Rehearse merge tradeoffs in the “Would you merge this?” lab before rung 5 touches production.
The longer explainer, with the episode clips, is on Chain of Thought. It draws on this episode.
Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.
Drafted with AI assistance from the episode transcripts.
Top comments (1)
Framing it as the cost of a wrong step rather than how smart the model looks is the right order. One thing a ladder can hide is that the rungs aren't independent: an agent that both writes and reviews its own code makes rung two weaker than it looks, since the same blind spots show up twice. Would you require a different model or a human for review once the agent is allowed to write?