The first version of our agent pipeline let the coding agent run the git and deploy steps. It didn't fail often, but when it did, it failed in a different way each time, and I couldn't debug that. Today roughly 90% of the pipeline is plain scripted code, and the agent does one job.
I'm Panth, and I lead the software team at Oizom, where Envizom is our environmental monitoring platform. Two days ago I wrote the short version: the eight steps that run after a developer drags a ticket to Git Push. This is the long one. It follows one ticket through every column of the board and goes through each job we took off the agent, with the reason, then what it costs and what broke.
The first version
Before agents, our problems were ordinary ones. Envizom is several repos: hardware drivers, a simulator, a backend and a frontend. A new developer needed days to get a working local setup. Releases meant keeping track of which dev database, prod database, backend and frontend were pointed at each other, and as time went on I was the only person who knew every combination.
Coding agents made my developers faster. With an agent, one developer can keep three tasks in flight. The mistakes then moved out of the code and into the branches around it. A change landed on the wrong branch, or a fix only worked on one machine. I wrote about that in One stack per ticket.
In the first version of the pipeline, the agent did more of the operational steps. It did the engineering, and it also ran git and the deploy. Compared with what we run now, it was slower and more expensive, and it was worse at the tasks that need to be boring.
The failures taught me the most. A script that fails the same way twice is a bug you can fix: you read the log, find the line and change it. An agent that merges a little differently on each ticket gives you nothing to fix. Each failure is a new story, and the next run will tell another one.
The split we ended up with follows one line. If a step is the same on every ticket, it is a script. The agent keeps the engineering.
One ticket, end to end
The board has seven columns. One of them has the agent in it.
| Column | Who acts | What happens |
|---|---|---|
| Feedback | developer | writes what they want, in plain English. That is the whole spec |
| Approved | scripts | dragging the ticket here is the handoff. The orchestrator provisions the ticket's stack |
| Working | agent | does the engineering work inside that stack, and talks only through the ticket thread |
| QAQC | developer | opens the preview the agent posted, tests it, replies in the thread or moves it on |
| Git Push | scripts | commit, merge dev in, open and merge a pull request per repo, post the links |
| VM Update | scripts | wait for CI, roll the new images onto the dev VM, health checks, rollback if unhealthy |
| Completed | scripts | the developer gets a notification, and the ticket's stack is torn down |
Each developer has their own devbox on our dev VM: VS Code in the browser, with our orchestrator, orch, inside it. Every developer is paired with their own agent, and their orch only picks up tickets assigned to that agent. The board is shared. The machines and the agents are not.
Here is a real ticket from the walkthrough video. I asked for a camera layout change on a device page. The agent built it in the ticket's stack and posted the preview. I replied in the thread: "also handle more than 4 cameras with a horizontal scroll, and set up 8 test cameras on this device". The ticket went back to the agent, which picked up with the context it already had, did both, and created the 8 test cameras in the ticket's own copy of the database. I tested the new preview and dragged the ticket to Git Push. After that, two notifications, and the change was live on dev.
The person in that story touched the ticket three times: to write it, to reply to a preview, and to drag it. The agent had one column. Scripts ran the other four.
The ten jobs scripts own
This is the list no agent touches: watching the board, provisioning the stack, copying the database, issuing certificates, driving git, opening pull requests, waiting on CI, deploying, health checks and rollback. They fall into two groups, before the agent starts and after a person says yes.
Before the agent starts
Watching the board. Orch watches for tickets dragged to Approved and assigned to its agent. Nothing here needs any thought. A ticket is either in the column or it isn't, and a model would add cost and nothing else.
Provisioning the stack. Every approved ticket gets a fresh branch in every repo, named after the ticket, plus its own backend, drivers, simulator and frontend. This is the step everything else depends on, so it has to be the same every time. If the stack came out a little different per ticket, nothing built on top of it could be trusted.
Copying the database. Each ticket gets a copy of the dev database, so it can change data freely. This is the safety decision in the whole design. An agent that runs a bad migration or deletes the wrong rows does it to the ticket's copy. Nobody else's work is in that box, so nobody else's work is touched. A step that sets up the safety net shouldn't be done by the thing the net is for.
In Envizom V3, which we are rebuilding from scratch, the copy is cheap. Every store has a dev driver. Postgres, S3, the time-series database and Redis all become SQLite in dev, and MQTT becomes a service built into the Go backend. SQLite runs inside the Go process, so copying a database means copying a few files. That change made our sandboxes about 10x cheaper and faster than sandboxes with full databases.
Issuing certificates. Every ticket has a private preview URL with an HTTPS certificate issued on demand. That URL is what the developer tests in QAQC, so it has to exist and work before the agent posts it.
After a person says yes
When the developer drags a ticket to Git Push, no agent is involved from that moment on. Orch runs eight steps:
Git Push
1 commit the work on the ticket branches
2 merge the current dev branch into them
3 open a pull request in each affected repo
4 merge the pull requests
5 post the PR and commit links into the ticket thread
VM Update
6 wait for CI
7 roll the new images onto the dev VM
8 run health checks against the VM
unhealthy -> roll back to the previous images, revert the merges,
tell the thread what happened
healthy -> Completed, notify the developer, tear the stack down
Driving git and opening pull requests. One of our goals from the start was that using agents would not weaken the normal process. Every feature still gets its own branch, commits, pull request and review. That only holds if those steps happen the same way on every ticket. An agent deciding how to merge is not a feature. A script that always merges the same way is.
Posting the links. Step 5 looks small, but it is how the thread becomes the audit trail. The PR and commit links land next to the ask, the previews and the feedback, so one ticket holds the whole story of one change.
Waiting on CI. Waiting is the clearest case of a job a model shouldn't hold. Nothing gets decided while it waits, and an agent session kept alive just to wait is still a session.
Deploying, health checks and rollback. These are the steps where sameness matters most. If the VM is unhealthy after the deploy, orch rolls it back to the previous images, reverts the merges and tells the thread, without anyone asking. A rollback that improvises is the last thing you want on the day something breaks. A scripted one runs the way it ran on every ticket before.
All of it stops at the dev environment. Auto-merge, the VM update and the rollback apply to dev only. Production goes through a separate path, where people make the call on a slower cadence. That is on purpose. We don't want automation deciding what reaches production, and we don't want a fast feature list to become the goal.
Taking away "I'm done"
The same split shows up in ticket-tracker, the board I open-sourced, where a small orchestrator turns each ticket into a headless Claude Code session. (The per-ticket stack scripts are not in it. They are Envizom-specific code.) It takes a few more jobs off the agent, ones that are easy to miss because they don't look like jobs.
Deciding it is finished. The agent never moves its own ticket. Its last message is structured output, enforced with --json-schema:
const OUTCOME_SCHEMA = {
type: 'object',
properties: {
status: { type: 'string', enum: ['review', 'waiting', 'blocked'] },
summary: { type: 'string' },
question_id: { type: 'string' },
},
required: ['status', 'summary'],
};
The orchestrator reads it. review moves the ticket to Review. waiting means a person is needed, so nothing runs until someone answers. blocked usually means the turn ran out of room, so the orchestrator resumes it by itself, up to 3 times, and then says on the ticket that it has stopped. The agent reports. The script decides what that means for the board.
Saying it is alive. The heartbeat comes from the orchestrator, not the agent: one beat every 60 seconds while the agent works. A crashed process can't report that it crashed, so health is worked out from the age of the last beat. The agent's part is its task list, which the orchestrator reads for the status message. More in Agent health from a 60-second heartbeat.
Working out what a turn cost. When a run ends, the orchestrator reads Claude Code's result line and posts the turn's receipt on the ticket. The app's comment box never posts one. Each receipt goes out with an idempotency key made from the ticket and the run number, so a retried request doesn't count a turn twice.
Waiting for an answer. When the agent needs a decision, it asks in the ticket and ends its turn. It is told not to poll. My answer starts the same session again with --resume. Headless Claude Code asks, exits, then resumes on the answer has the details.
Any of these could be left to the agent. Each one is the same on every ticket, so none of them is.
What the agent still does
The engineering. It reads the ticket, works in the ticket's own repos and services, changes the code, and creates the test data it needs in the ticket's database copy, as in the 8 test cameras. When the developer replies with feedback, it picks up with the same context it had before, without re-reading the repo to find out where it was.
It also asks. A real decision that belongs to the owner, such as scope, design, or anything destructive or outward-facing, becomes a question in the ticket rather than a guess.
That is the 10%. It is the only part of the pipeline where the right answer changes from ticket to ticket, so it is the only part where a model is worth what it costs.
What stays with people
Writing the ticket. The thread is the only way to talk to the agent, so every prompt is stored with the work it produced. I read how each person on the team asks for things and coach them on it. That has done more than any prompting guide. See Our coding agent prompts live in the ticket.
QA. The agent posts a preview and moves the ticket to QAQC, and that is as far as it goes. The developer who asked tests the preview, because they are the only one who knows what they meant. An agent grading its own work would remove the one review that matters most. Every step after QA is built to be boring and repeatable, and none of them asks whether the change was the right change. QA is where that question gets asked, so it is the one place we didn't automate. More in 1 human check, 8 scripted steps.
The drag to Git Push. Shipping to dev is a person's action in the thread. Shipping to production is a separate, slower decision.
What it costs
| What | Number | Source of the number |
|---|---|---|
| One agent turn in the walkthrough video | $4.77 | the receipt under the agent's reply: "turn $4.77 · total $4.77" |
| The dev VM with every devbox and ticket stack | about ₹4–6k a month | rented, one machine |
| Tickets it runs in parallel | 6, comfortably | same VM |
| Sandbox cost after SQLite dev drivers | about 10x cheaper and faster | V3 rebuild |
| A scripted step | no turn receipt | it is not a model call |
The last row is the plain reason scripts are cheaper. Every agent turn posts its cost into the ticket thread, right under the answer. A scripted merge, deploy or health check posts no receipt, because it doesn't call a model. An operational step done by the agent is done inside a billed turn.
The receipt is a design choice too. The person who asked sees what the ask cost, where they asked it, and it changes how they write the next one.
What broke on the scripted side
Scripts break as well. The difference is how they break.
The cost number was wrong. On ticket-tracker, one ticket reached $402 over nine agent runs and nothing in the app said so. The orchestrator already wrote the cost of every run to a local log, but nothing showed it where the work was. Putting the number on the ticket had a trap: Claude Code's total_cost_usd is cumulative across --resume. Run 8 made one API call with zero output tokens and reported $392.83, the same as run 7. The fix is a few lines in the orchestrator. It keeps the last total per session and posts the difference. The full fix is in $402 on one ticket.
Polling was most of the bill. Two orchestrator loops asked the tracker's API for the full ticket and event lists every 30 seconds or so. Over three days that was 8,822 + 8,815 calls, 88% of every request the API served. The fix was again in the script. Agents now hold one stream on a small revision node and fetch only the delta when it moves. That one is in Polling was 88% of my API calls.
The heartbeat was on the bill twice. The first version kept each beat stream as a Firestore document: 1,440 writes per agent per day, plus a sweep that ran every minute to notice silence, 4,261 runs in three days. Beats moved to the Realtime Database, about 100 bytes each, and silence is now worked out at read time from the age of the last beat.
None of these needed an agent to diagnose. Each showed up in a log or a bill, and each was fixed in one place in the code. That is the trade I wanted from the split. The agent's mistakes stay inside a ticket's stack, where a bad change hit a copy. The scripts' mistakes show up in a log, the same way twice, so they can be fixed.
Why the boring 90% makes the 10% useful
Most of what makes the pipeline worth using comes from the scripted part. Per-ticket isolation is what lets the agent work freely, and lets a ticket sit in QAQC for a week while three others ship. The thread is the audit trail because scripts post into it at every step: the preview, the cost of each turn, the PR and commit links, and the rollback if there is one. Stored prompts are reviewable because the thread is the only channel. Human QA and dev-only scope are the two places we chose not to automate, and both are easier to hold when everything around them is predictable.
The order I'd build it in
If you're setting up something like this:
- Isolation first. If every task can't run in its own branch, with its own services and its own data, nothing after it is safe.
- A handoff surface the team already uses. For us that was the ticket.
- Script every step that is the same on every ticket: provisioning, git, CI, deploy, health, rollback, and the receipts.
- Only then, the agent inside the box, with the smallest job you can give it.
The ticket is the interface has the walkthrough video and the first write-up of the pipeline.
If your team runs coding agents, which step is the agent doing today that a script would do the same way every time?
I lead the software team at Oizom, where Envizom turns data from air-quality and environmental monitors in 80+ countries into something cities and industries can act on. I dropped out before the AI wave, because I figured the engineering was in solving problems in the real world, and my college made me choose between attendance and real-world applications. Find me on LinkedIn.
Top comments (0)