A fallback chain that exists only in conversation is not a fallback chain; it is a hope with a diagram attached. The pairing session below started from that sentence and ended with something small and unglamorous: one JSON file, a short resolver, and a single regression test. None of it makes a network call, which is exactly why the chain was accepted before the first request left the laptop. The kept decision fits in one line: every fallback must be declared in data, resolved on a dry run, and pinned by a test that fails when the declaration weakens.
The constraint that opened the session
The senior joining the session refused to read the application code first, and asked instead for an answer to a narrower question about behaviour under failure. That refusal set the shape of the hour, because the discussion stayed at the level of declared intent rather than implementation cleverness. The reasoning offered was blunt and, in hindsight, correct: a chain that cannot be described in a table cannot be reviewed in a diff. Everything that followed was an attempt to make the description cheap enough to keep.
The session ran in the middle of a loud general argument about AI replacing developers. It treated that argument as background noise and asked a smaller question instead: when the model call fails, what does the system do, and who decided that.
The three questions, in the order asked
- Which steps exist, and what is each one for? The senior wanted a purpose string per step, short enough to fit on one line and specific enough to disagree with.
- What class of failure moves the request to the next step? Timeout, refusal, malformed output, and rate limit are different problems that deserve different responses.
- Who pays for step two when step one was free, and is that approval recorded anywhere? Unrecorded cost is how a free prototype quietly becomes an invoice.
Each question maps to one field in the data file. That mapping is the entire design, and it is boring on purpose, because boring declarations survive reorganisation.
Two dead ends, recorded so they are not repeated
The first dead end was a retry loop dressed up as a fallback. Asking the same endpoint three times with the same prompt returns the same refusal most of the time, and the loop hid that fact inside latency. The senior called it the same door, knocked louder, which ended the discussion faster than any benchmark could have.
The second dead end was a smart router that picked a step based on prompt length. Nobody could state the threshold, nobody could test it, and the rule silently changed behaviour between environments. The pair dropped it and kept an explicit ordered list instead, because order is a decision a reviewer can actually disagree with.
The kept decision, stated plainly
Fallbacks are declared as ordered data, validated locally, and exercised in a dry run before any real traffic reaches them. The chain is allowed to be short, and a chain of length one is a legitimate answer when a team accepts the failure mode. What is not allowed is a fallback that first appears in an incident review.
Step 1: write the chain as data
The file below describes three steps without naming a vendor, which keeps the review honest about what is actually being promised. The on_failure field carries the decision, and cost_class makes the free step visible next to the one that costs attention.
{
"schema": 1,
"steps": [
{
"id": "primary",
"kind": "model",
"purpose": "default generation for reviewed patch summaries",
"on_failure": "fallback",
"cost_class": "free",
"timeout_s": 20
},
{
"id": "secondary",
"kind": "model",
"purpose": "same prompt, different endpoint, no prompt rewriting",
"on_failure": "escalate",
"cost_class": "free",
"timeout_s": 30
},
{
"id": "human",
"kind": "human",
"purpose": "reviewer narrows the task or defers it to the next day",
"on_failure": "stop",
"cost_class": "attention",
"timeout_s": 0
}
]
}
Step 2: resolve it without spending anything
The resolver reads the file, checks required fields, rejects duplicate ids, and refuses any paid step that lacks recorded approval. It prints the order so a reviewer can read the chain aloud during review, which is the cheapest correctness check available.
#!/usr/bin/env python3
"""Resolve a declared fallback chain without making a single network call."""
import json
import sys
from pathlib import Path
REQUIRED = ('id', 'kind', 'purpose', 'on_failure', 'cost_class')
VALID = ('fallback', 'escalate', 'stop')
def load(path):
data = json.loads(Path(path).read_text())
return data.get('schema'), data.get('steps', [])
def check(schema, steps):
problems, seen = [], set()
if schema != 1:
problems.append('unexpected schema: %r' % (schema,))
if not steps:
problems.append('chain is empty; state the single step explicitly')
for i, step in enumerate(steps):
missing = [f for f in REQUIRED if f not in step]
if missing:
problems.append('step %d (%r) missing %s' % (i, step.get('id'), missing))
continue
if step['id'] in seen:
problems.append('duplicate id: %r' % step['id'])
seen.add(step['id'])
if step['on_failure'] not in VALID:
problems.append('step %r has unknown on_failure' % step['id'])
if step['cost_class'] == 'paid' and not step.get('approved_by'):
problems.append('paid step %r has no approved_by' % step['id'])
return problems
def main(path):
schema, steps = load(path)
problems = check(schema, steps)
order = ' -> '.join(s.get('id', '?') for s in steps)
print('resolution order: ' + order)
for step in steps:
print(' %-10s %-8s %s' % (step.get('id'), step.get('on_failure'), step.get('purpose')))
for problem in problems:
print('PROBLEM: ' + problem, file=sys.stderr)
return 1 if problems else 0
if __name__ == '__main__':
sys.exit(main(sys.argv[1]))
Step 3: pin the decision with a test
The test asserts success for the good file and failure for a weakened copy, so deleting a decision now breaks the build instead of surfacing months later. The second case matters more than the first, because it is the one that catches a quiet edit made during a busy week.
import json
import subprocess
from pathlib import Path
CHAIN = Path('chain.json')
def run(path):
return subprocess.run(['python3', 'route_check.py', str(path)], capture_output=True, text=True)
def test_declared_chain_resolves():
result = run(CHAIN)
assert result.returncode == 0, result.stderr
assert 'primary -> secondary -> human' in result.stdout
def test_chain_without_failure_mode_is_rejected(tmp_path):
data = json.loads(CHAIN.read_text())
del data['steps'][1]['on_failure']
broken = tmp_path / 'broken.json'
broken.write_text(json.dumps(data))
assert run(broken).returncode == 1
Both checks run in CI with one line, which keeps the guard cheap enough that nobody argues about it during review:
python3 route_check.py chain.json && pytest -q test_route.py
Step 4: keep the pairing log next to the code
A short log file, one entry per session, records the question, the dead end, and the decision that survived into the repository. Future diffs then reference a decision instead of a mood, and reviewers spend their attention on ordering rather than archaeology. This habit costs about two minutes and prevents the same argument from being re-run a quarter later, which is the usual outcome of undocumented routing choices.
Where free model access changes the arithmetic
The pattern above is only viable if trying it costs nothing, and this is the narrow place where free model access matters for a small team. A free hosted endpoint lets the pair run the declared chain end to end, watch real failure classes appear in the log, and then decide whether the ordering still makes sense. The project's free server option supports the same habit, since a service that must be reachable during a review should not require a procurement thread just to stand up. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Current quotas, available models, and server terms belong on the project page rather than in a blog post, because those numbers change faster than articles do.
Limitations and who should skip this
The dry run validates declarations, not behaviour, so it cannot predict provider-side semantic drift or rate-limit interactions between steps. Prompt sensitivity across endpoints is real, and the resolver will happily approve a chain whose second step behaves badly on the same input. Teams with a single endpoint and an accepted failure mode should keep one step and skip the mechanism entirely. Organisations that cannot agree on cost classes will produce a table that reads well and means nothing, which is worse than having no table at all.
The exit condition
The session ended when the senior could read the order aloud, name the failure class between steps, and point at a test that would fail if either fact changed. That took under an hour and produced roughly sixty lines that will outlive several model upgrades. Anyone copying the pattern can start with the data file and the resolver, then point the first real call at a free endpoint to see which failure classes actually appear. The worthwhile part is the declaration, not the vendor.
Top comments (1)
My fallback chain looked fine in the diff and had never once run. What caught it was forcing each step to fire on a real request; two of the three steps were broken in ways no unit test saw, because the test called them directly instead of going through the failure. Declaring the order is the cheap part. Forcing the failure through the same door users come in is the part that pays.