Most developers working with coding agents still interact with them directly. They open a terminal, prompt an agent, and often run several agents at once. This works well, but deploying agents at greater scale raises new challenges. How do you coordinate work across the development lifecycle, ensure agents produce reliable software, and maximize how much your team ships without letting costs spiral?
I often see people suggest that setting up a software factory is as simple as stringing together prompts and cloud agents. In my experience, that’s not enough. What works for a handful of workflows doesn’t hold up in production. Companies need systems that are reliable, measurable, configurable, and capable of improving over time.
I call the discipline of building these systems factory engineering: bringing engineering rigor to deploying coding agents at scale so teams can ship more, better software at the lowest possible cost.
What Is a Software Factory?
A software factory is a code-defined system for coordinating and continuously improving cloud coding agents across the software development lifecycle. Agents can triage incoming issues, break complicated requests into specifications, implement changes, and send them through review and verification. Once work ships, monitoring provides feedback that can improve future runs.
The important distinction is that a factory needs to be engineered as a system, not assembled as a loose collection of automations. Engineers define how work moves through it, where agents make decisions, and where people get involved. Defining factories in code makes the system easier to inspect, measure, and improve.
A production factory also requires several layers: a way to bring work in, a control plane to distribute it, an execution environment where agents run, and a data layer that captures what happens. Each requires engineering. You need to consider how work gets routed, what tools and context agents can access, and what happens when an agent fails midway through a task.
Engineering and Measuring the Factory
One of the biggest mistakes in the conversation about agentic development is relying too heavily on opinions about which model or coding harness is best. Public benchmarks can be useful, but they don’t tell you how well a particular setup will perform on your codebase or workloads. You need to measure based on your own data.
A factory engineer should run different configurations against the team’s own tasks and compare the results. If you change a model, harness, or workflow, you want to know whether that change actually improves performance. Benchmarks and A/B tests make those comparisons possible.
At Warp, we look at metrics - such as cost per pull request, throughput, and revert rate. We want to understand how much software we’re shipping, what it costs, and whether the changes hold up. Agent performance varies depending on the work, the codebase, and how the system is configured. What works elsewhere may not work for your team.
Factory engineers also need tools to continuously improve the system. Blueprints define the factory and how work moves through it. Automations handle development steps, while evals measure agent performance and identify problems. Self-improvement allows the factory to use what it learns from previous runs to improve future ones.
For example, imagine a code review agent that repeatedly makes comments a senior engineer has to correct. Those corrections can be captured and used to improve the agent’s review skills, so the next run benefits from what the engineer learned in the last one. Rather than treating every agent run as an isolated experiment, the factory can learn from its performance and get better over time.
What This Means for Engineers
I still find product engineering motivating. I like figuring out what customers want and building something useful. Factory engineering doesn’t take that away. It gives engineers more leverage to focus on the product by taking on more of the implementation work that previously consumed their time.
As agents become more capable, engineers will spend less time writing code and managing individual agent interactions. More of the work will involve designing the systems that coordinate agents, measuring their performance, and improving how they operate. Human judgment remains essential: the factory needs people who understand the product, know what good software looks like, and decide what should be built.
For engineers who take pride in the craft of building, this shift could feel like a loss of control. In my experience, though, the engineers most energized by this work tend to be senior engineers who enjoy hard systems problems, like optimizing and debugging performance issues and figuring out how to make the whole system work better.
The next frontier of software engineering isn’t just getting better at working alongside coding agents. It’s bringing the same engineering discipline we apply to building software to the systems that build it.
Zach Lloyd is the founder and CEO of Warp, a platform for open agentic development used by leading AI labs, Big Tech, and over half of the Fortune 500. Before founding Warp, he spent nearly a decade as Principal engineer on the Google Docs suite, helping scale products like Sheets to hundreds of millions of users.
Top comments (0)