We built a nine-agent pipeline that turns a natural language use case into working code, an interactive preview, and an implementation guide in under 4 minutes. It serves roughly 800 to 1,000 users per day, but its first production version consumed about 30,000 tokens and made 15 to 20 Pro-tier model calls per request. The useful lesson from this multi-agent AI implementation is not simply that more agents can solve a larger problem. It is that production reliability comes from deciding which work belongs to a model, what context must be mandatory, and where strict contracts should replace freeform generation.
Decompose work around failure boundaries
The system needed to analyze a requested use case, find relevant APIs, generate visual configuration, write code, inspect that code, repair defects, and produce implementation documentation. Putting all of those responsibilities into one prompt would have created an oversized context, less predictable output, and no clean place to isolate failures.
We divided the workflow into nine agents. A root agent manages session state, routing, and runtime instruction composition. Specialized agents handle analysis, styling, code generation, evaluation, refinement, and documentation. Each receives a narrow responsibility, an explicit input, and an expected output.
That separation matters because it creates operational boundaries. A style configuration can be validated without rerunning code generation. An evaluator can reject unsafe code without rewriting unrelated sections. Each stage can be tested, tuned, and assigned a model based on the work it performs.
Decomposition also carries a cost. Every boundary introduces orchestration, state transfer, and another potential failure point. The architecture was appropriate because the workflow already contained distinct tasks with different validation rules. Splitting an arbitrary prompt into several agents would only distribute the ambiguity.
A practical criterion is to look for independently testable failure modes. If a task can be given a clear contract and evaluated without judging the entire result, it may deserve its own stage.
Use deterministic code when probability adds no value
Two stages, the Schema Validator and Documentation Builder, make no large language model calls.
The Schema Validator performs deterministic checks on generated configuration. The Documentation Builder detects APIs in the final code and maps them to documentation URLs from a canonical registry. Neither task benefits from linguistic creativity. Implementing them programmatically removes 2 possible sources of hallucination and reduces token consumption for those stages to zero.
The same principle shaped the boundaries between code generation, evaluation, and refinement. Asking one agent to generate code and then approve its own output creates a weak feedback loop. We separated the evaluator from the refiner. The evaluator applies strict rules and security scanning; the refiner receives specific findings and makes targeted corrections.
Every agent that generates code or configuration must also satisfy a structured output schema. This prevents freeform commentary from leaking into machine-consumed output, reduces unnecessary tokens, and gives downstream agents parseable input without an extraction step. More importantly, each handoff becomes independently validatable.
Models are useful where interpretation or synthesis is required. Validation, lookup, and contract enforcement should remain ordinary software when their rules can be expressed directly.
Make critical context mandatory
The Model Context Protocol, or MCP, gives the analysis, code generation, and code evaluation agents access to live API documentation through an indexed knowledge base. Query results are capped to control context growth, and the MCP server runs as an isolated service so documentation retrieval does not compete with model calls for API quota.
This keeps knowledge separate from agent code. Documentation can change without requiring the agents to be redeployed.
Retrieval alone did not solve the correctness problem. MCP tool calls are initiated by the agent, so a confident model can decide that it already knows an API and skip the lookup. That creates a path to plausible but invalid code.
We addressed that gap with dynamic instruction injection. The analysis stage identifies which APIs a request needs. Canonical examples for those APIs are then inserted directly into downstream instructions. Open-ended questions still use MCP retrieval, while required implementation patterns arrive as mandatory context.
This distinction is broadly useful. Use optional retrieval for information the model may need to explore. Inject context directly when correctness depends on the model seeing it. After moving from static examples in every prompt to relevant examples selected at runtime, the system reduced per-request token consumption by 73%.
Optimize from production evidence
The first release used Gemini 2.5 Pro for code generation, evaluation, and refinement. It worked, but using the most capable model throughout the pipeline made both cost and latency unsustainable at production volume.
The system evolved through five engineering phases. Dynamic instruction injection removed irrelevant prompt material. Model tiering assigned work according to task requirements and made evaluation cycles roughly 70% faster, with no measured loss of quality. Caching and budget controls reduced repeated work, while BigQuery telemetry and structured observability made the effects measurable.
Reliability work extended beyond average latency. The pipeline includes security auto-fix loops, graceful fallback to the previous valid output, and three-layer retry behavior. Its environments are managed with Terraform across development, staging, and production, with automated CI/CD gates and Google Cloud services including Vertex AI Agent Engine, Cloud Run, Secret Manager, and Cloud Logging.
The transferable method is to begin with explicit stage contracts, observe production behavior, and optimize the stages that evidence identifies. Model choice, prompt context, retries, caching, and deterministic code are all controls within the same system. Treating them that way turns an agent demo into a service that can deliver consistent results at daily production volume.
Top comments (0)