Meta's data pipeline agents automate column onboarding across three code repositories, three languages, and six pipeline stages. The first version used a single agent loaded with all domain knowledge. It hallucinated in production. The fix was not better prompts or larger context windows. It was giving the orchestrator less to do.
The thin orchestrator pattern separates routing logic from domain reasoning. The orchestrator enforces reliability checks and delegates work. Specialized agents handle schema inference, query generation, and validation. This inverts the common pattern of stuffing all context into a single orchestrator prompt.
Why the Fat Orchestrator Failed
Meta's pipeline is config-as-code. Python configurations drive C++ code generation, which feeds data processing services, which populate analytics systems. Onboarding a single data column requires coordinated changes across configuration registries, backend services, and data pipelines.
The first agent design loaded the orchestrator with:
- Full schema definitions for three code repositories
- Language-specific syntax rules (Python, C++, SQL)
- Pipeline stage dependencies and sequencing constraints
- Validation rules for each stage
- Error recovery procedures
In demos, it worked. In production, it started generating invalid configurations. The root cause was context overload. The orchestrator tried to reason about domain semantics while also managing workflow state. When schema definitions changed or new edge cases appeared, the agent's routing decisions became brittle.
The Thin Orchestrator Architecture
Meta rebuilt the system with a 1,300-line orchestrator that routes 20+ workflows to 14 specialized agents. The orchestrator performs zero domain work itself.
Orchestrator Responsibilities
The thin orchestrator owns three things:
- Workflow sequencing: Which agent runs next based on current state
- Reliability checks: Timeout enforcement, retry logic, error boundaries
- State management: Passing outputs from one agent to the next
It does not interpret schemas, generate code, or validate domain logic. It does not need to understand what a "data column" is or how pipeline stages relate.
Specialized Agent Responsibilities
Each specialist owns one domain task:
- Schema inference agent: Reads source data, proposes column types
- Config generation agents: One per language (Python, C++, SQL)
- Validation agents: Check syntax, type consistency, pipeline compatibility
- Deployment agents: Apply changes, verify rollout
Specialists receive narrow context. The schema inference agent sees only the source data and existing column definitions. The Python config generator sees only the target schema and Python syntax rules. No agent needs the full system picture.
Routing Without Domain Knowledge
The orchestrator routes based on workflow state, not task semantics. It uses a simple decision tree:
class ThinOrchestrator:
def route_next_agent(self, workflow_state):
if workflow_state.stage == "schema_inference":
if workflow_state.schema_proposed:
return self.agents["validation"]
else:
return self.agents["schema_inference"]
elif workflow_state.stage == "config_generation":
target_lang = workflow_state.target_language
return self.agents[f"config_gen_{target_lang}"]
elif workflow_state.stage == "deployment":
if workflow_state.validation_passed:
return self.agents["deployment"]
else:
return self.agents["error_handler"]
else:
raise UnknownStageError(workflow_state.stage)
The orchestrator checks flags (schema_proposed, validation_passed) and routes accordingly. It does not inspect the schema content or validation results. It trusts specialists to set the correct flags.
Reliability Checks Without Semantics
The orchestrator enforces reliability without understanding task semantics:
- Timeout boundaries: Each agent call has a 30-second timeout
- Retry logic: Failed calls retry up to three times with exponential backoff
- Error isolation: Agent failures do not crash the orchestrator
- State snapshots: Workflow state is persisted after each agent completes
These checks apply uniformly across all agents. The orchestrator does not need to know what "schema inference" means to enforce a timeout on the schema inference agent.
Trade-offs and Failure Modes
| Aspect | Thin Orchestrator | Fat Orchestrator |
|---|---|---|
| Context per agent | Narrow, task-specific | Full system knowledge |
| Routing logic | State-based flags | Domain reasoning |
| Failure isolation | Agent-level boundaries | Orchestrator collapse |
| Prompt maintenance | Distributed across specialists | Centralized, brittle |
| Debugging | Trace agent outputs | Inspect orchestrator reasoning |
| Latency | Higher (more agent calls) | Lower (fewer hops) |
The thin pattern trades latency for reliability. Each workflow makes more agent calls because specialists do not overlap. A fat orchestrator might generate and validate a config in one step. The thin version calls the config generator, then the validator, then checks the result.
The thin pattern also shifts debugging. When a workflow fails, you trace which agent set the wrong flag. With a fat orchestrator, you debug why the orchestrator made the wrong routing decision, which often means inspecting its internal reasoning about domain semantics.
Production Metrics
Meta reports efficiency and accuracy improvements after switching to the thin orchestrator:
- Hallucination rate: Dropped from 12% to under 2%
- Workflow success rate: Increased from 78% to 94%
- Median latency: Increased by 1.2 seconds (acceptable for batch onboarding)
- Prompt maintenance: Reduced by 60% (specialists have smaller, more stable prompts)
The latency increase is real but acceptable for Meta's use case. Data column onboarding is a batch process, not a user-facing API. The reliability gains outweigh the extra round trips.
When the Orchestrator Should Stay Thin
The thin pattern works when:
- Workflows span multiple domains with distinct knowledge boundaries
- Reliability matters more than latency
- Domain logic changes frequently (specialists update independently)
- You need clear failure attribution (which agent broke the workflow)
The thin pattern does not work when:
- Workflows require tight coupling between steps (agents need shared context)
- Latency is critical (every agent call adds overhead)
- Domain boundaries are unclear (hard to split specialists cleanly)
- You have fewer than five distinct tasks (overhead of managing specialists exceeds benefit)
Implementation Notes
Meta's orchestrator uses a state machine with explicit transitions. Each agent returns a structured output that includes:
- Result payload: The domain work (schema, config, validation report)
- Next stage flag: What workflow stage to enter next
- Error signal: Whether the agent succeeded or needs retry
The orchestrator does not parse the result payload. It only checks the next stage flag and error signal. This keeps routing logic independent of domain semantics.
Specialists communicate through a shared state object, not direct calls. The orchestrator mediates all data flow. This prevents specialists from developing hidden dependencies.
Technical Verdict
Use the thin orchestrator pattern when you have multi-domain workflows where reliability and maintainability matter more than latency. It works best for batch processes, data pipelines, and infrastructure automation where each step has clear inputs and outputs.
Avoid it for latency-sensitive user-facing APIs or workflows where steps need shared context to make decisions. The overhead of routing through a central state machine adds real latency, and forcing specialists to communicate only through flags can create awkward indirection.
The key insight is that orchestrators should route, not reason. When your orchestrator starts needing domain knowledge to make routing decisions, you have crossed into fat orchestrator territory. Pull that knowledge back into specialists and let the orchestrator stay thin.
Top comments (0)