A discovery stage needs source tools, a defined reasoning profile, and a way to preserve the evidence it finds. Its terminal artifact might be a source packet with explicit claim boundaries.
Now imagine that the primary provider times out. The orchestrator moves to the next model in a generic fallback list, but that route cannot use the required source tool. The setting is ignored, or the workflow quietly switches to text-only generation. A polished packet appears and the run turns green.
The workflow stayed available by changing the job. Fluent prose makes the failure harder to spot because every later stage receives an artifact that looks valid but was produced outside the discovery contract. If the dashboard records only "completed," that mismatch can survive all the way to the final output.
The model does not have to be bad for this to happen. The routing policy only has to treat unlike jobs as interchangeable. A reliable router defines each stage's contract, selects a route that can satisfy it, rejects incompatible substitutes, and records what actually ran. Availability fallback fits inside that policy; it does not get to override it.
One workflow contains several different jobs
"Use the best model" sounds decisive until a pipeline has several stages. Best at what? Discovery may need browsing or retrieval tools, enough context for the evidence set, and evidence capture. Planning works on a bounded packet and turns it into an argument. Drafting has platform, voice, length, and file constraints, while evaluation needs explicit checks tied back to the brief and sources.
Reasoning effort belongs in the route definition as well. It is a budget and behavior setting for a particular task, rather than a global quality switch that should automatically be pushed to maximum.
Current SDKs expose the seams needed to make those choices explicit. The OpenAI Agents SDK documents model and provider selection per agent, configurable reasoning effort, and strict validation for unsupported features. Vercel's AI SDK documents step preparation using runtime context, including model selection, along with reasoning controls and telemetry. Vercel's routing guide separates task-oriented routing from cost, latency, fallback, load balancing, and budget policies.
Those controls do not prove that every pipeline needs multiple models. Often one route is enough for the whole workflow. The useful change is that reuse becomes a checked decision instead of an inherited default: if the same route satisfies all four contracts, assign it four times and record the choice. The design is about preserving each job, not collecting model names.
Start with a four-stage contract
Build the route table from the jobs before adding model names. Here is a practical four-stage design for a source-bounded publishing workflow. It is an implementation example, not a benchmark claim about which kind of model performs best.
| Stage | Required capabilities | Terminal artifact | Route rejection condition |
|---|---|---|---|
| Discovery | approved source tools, enough context for the evidence set, source evaluation and evidence capture | source packet with claim boundaries | required tool, context, or capture behavior is unsupported |
| Planning | bounded source-packet input, synthesis, argument and outline structure | source-linked argument and outline | route or stage setup cannot preserve the evidence boundary |
| Drafting | source-packet input, platform format, voice and length constraints, file output | non-empty draft at the expected path | route cannot preserve input, formatting, or artifact requirements |
| Evaluation | brief and source access, explicit checks, structured pass/fail result | evaluation record linked to the draft | route cannot execute the required checks or return the agreed structure |
A useful boundary exists between the first two rows. Discovery gathers evidence with approved source tools, checks whether the material is relevant and readable, and records what each source can support. Planning receives that bounded evidence packet and decides the article's argument, order, and outline. It should not reopen broad discovery whenever it wants more material. If the packet has a real gap, the planner can return that gap for a deliberate discovery pass instead of silently widening its own job.
A compact contract can live in configuration rather than prompt prose:
version: 3
stage: discovery
task_class: source_bounded_research
requires:
tools: [approved_source_reader]
context_profile: research_packet
evidence_capture: true
reasoning_profile: deliberate_research
expected_artifact: research/source-packet.md
approved_routes:
- research-primary
- research-secondary
The profile aliases resolve to concrete provider, model, reasoning, and tool settings elsewhere. That keeps infrastructure policy in orchestration state, where the runtime can inspect it, while the prompt describes the work. Real contracts may also need budget or latency boundaries. A field earns its place only when the router can validate it or the evaluator can check it; anything else is decoration.
The stage-routing loop
The mechanism is small enough to implement without building an internal routing product.
Declare the contract before selecting a model. Record the task class, required tools and features, context needs, reasoning profile, budget boundary, expected artifact, and approved substitutes.
Resolve a route from the contract and current runtime context. Put the choice in code or versioned configuration so an application-wide default cannot make it by accident.
Validate compatibility before execution. Check tools, schema behavior, context, settings, and output support. OpenAI's
strict_feature_validation=Trueis one concrete example: an unsupported setting can become a visible error instead of being ignored. Other stacks will expose different checks, so the general rule is to test the requirements you depend on.Classify a failure before choosing recovery. A timeout, a missing capability, and repeated provider instability call for different responses.
Emit a receipt for the route that ran. Configuration shows intent. The receipt shows execution.
The order matters. When fallback selection happens before compatibility validation, a generic list can quietly weaken the contract.
Fail closed on capability, fail over on availability
"Fail closed" can sound rigid in a system designed to keep working. The useful question is what failed.
| Runtime event | Response |
|---|---|
| transient timeout or provider error | retry within a bounded policy, then try an approved compatible route |
| provider rejects the context size | use an approved larger-context route only if every other contract field still passes |
| required tool, schema, or setting is unsupported | reject that route; test another against the original contract or stop |
| repeated provider instability | open or observe a circuit according to policy |
| no approved compatible route | fail the stage visibly |
Retries handle transient attempts, fallbacks choose alternate execution routes, and circuit breakers stop sending traffic into repeated failure. Capability rejection happens earlier: it says that a route cannot perform this job at all. Flattening those mechanisms into one chain produces deceptive uptime, with more green runs and less certainty about what "completed" means.
The practical consequence is simple. A generic text model cannot cover tool-backed discovery in an emergency, and an evaluation route with different schema behavior needs validation before it can substitute. Even a larger context window is irrelevant if the route loses a required tool. Every replacement has to pass the current contract rather than merely produce an answer.
This fail-closed rule is the article's operational recommendation, derived from the routing and validation controls in the sources. Neither vendor mandates that exact policy wording.
Verify the runtime, not the configuration
Even a good route table cannot tell you what happened after a retry, runtime override, or provider failure. Evidence from the run closes that gap.
The OpenAI Agents SDK documents nested tracing across runs, agents, model generations, tools, guardrails, and handoffs. Vercel describes telemetry spanning the root generation, model calls, steps, tools, usage, errors, and selected context. Those surfaces make a compact routing receipt practical.
A useful receipt might contain:
stage: discovery
contract_version: 3
selected_route: research-primary
actual_provider: <provider-id>
actual_model: <model-id>
reasoning_setting: <resolved-setting>
compatibility_check: passed
recovery:
action: fallback
reason: provider_timeout
terminal_status: completed
artifact: research/source-packet.md
trace_id: <trace-id>
This schema is a recommendation, not a standard supplied by either SDK. Its job is narrow: show the selected and actual routes, explain any change, record whether the substitute passed the contract, and point to the terminal artifact.
A receipt cannot prove that the draft is accurate or good. It helps an operator separate routing failures, tool failures, and output-quality problems. Evaluation still has its own work to do.
Tracing also creates a data-handling problem because generation and function spans may include sensitive inputs or outputs. Capture only what diagnosis requires, redact deliberately, restrict access, and set retention rules. Recording everything by default is a liability, not an observability strategy.
The cost and complexity objection is valid
A router adds code, configuration, tests, and another place to fail. That cost is real, so start with the two stages that have the clearest capability difference. Define their contracts, keep a tiny approved route table, and add one preflight check for the requirement most likely to disappear during fallback. Record a redacted receipt, then force one provider failure and confirm that the substitute preserves the contract or stops.
Route only where capability, risk, or artifact semantics materially change. A second model adds no value when the first route already passes all four stage contracts; in that case, the simplest correct table contains one model with four explicit assignments.
Cost control belongs here too, without a promise that routing automatically saves money. Reasoning effort, model tier, retries, and budget ceilings are policy dimensions you can assign per stage and measure. Extra reasoning is not universally better, and a lower-priced route is a poor bargain if it corrupts the artifact consumed by every later stage.
The complexity has to buy a visible property: the workflow performs the declared job or reports that it could not. Without that property, the router is only a more elaborate fallback chain.
Reliability means preserving the job
It is easy to mistake an AI stage's output for evidence that the stage worked, especially when the output reads well. The safer sequence is to declare the contract, resolve and validate an approved route, recover according to the failure class, and record the runtime that produced the artifact.
Then ask the test that generic fallback chains avoid: if the primary provider failed today, could you prove that the fallback performed the same job, or would you only know that it produced plausible text?
Top comments (0)