Characterization tests should freeze current outputs before any refactor. The smallest safe change is one pure helper, not a rewrite. This plan locks fourteen webhook retry rows, then moves backoff math.
Start with the contract, not the cleanup
Messy retry functions often hide load-bearing caller bugs. A broad rewrite can change a delay callers already expect. Frozen rows make that dependency visible before any edit.
You do not need a perfect product spec to start. You need the outputs the current function already returns. Treat those outputs as a temporary contract, not as truth.
This walkthrough uses a webhook retry decider as the sample. The domain is delivery status, not payment capture. That limit keeps the exercise inside pure data.
Read the messy function first
The sample below is a labeled proposal, not a live trace. It mixes status checks, body sniffing, and delay math. No socket call belongs inside this decision function.
The listings were not run against a private repository. The numbers inside are teaching constants, not measured SLOs. Copy them only as a starting contract for your own fork.
def decide_retry(attempt, status, elapsed_ms, body):
text = (body or '').lower()
if attempt >= 5:
return {'retry': False, 'delay_ms': 0, 'reason': 'cap'}
if status in (200, 201, 204):
return {'retry': False, 'delay_ms': 0, 'reason': 'ok'}
if status == 429 or 'slow down' in text:
delay = 1000 * (2 ** attempt)
if elapsed_ms > 30000:
delay = min(delay, 2000)
return {'retry': True, 'delay_ms': delay, 'reason': 'rate'}
if 500 <= status <= 599 or 'timeout' in text:
delay = 250 * attempt
if attempt == 1:
delay = 400
return {'retry': True, 'delay_ms': delay, 'reason': 'server'}
if status in (408, 409) and elapsed_ms < 5000:
return {'retry': True, 'delay_ms': 750, 'reason': 'conflict'}
return {'retry': False, 'delay_ms': 0, 'reason': 'drop'}
Three branches share one return shape, which helps tests. The rate-limit branch also clamps delay after long waits. That clamp is easy to drop during a cleanup.
Attempt five or higher always returns a hard cap. Success statuses never retry, even with odd body text. Unknown statuses drop, which may be a product bug.
Branch order is part of the observed behavior. A 429 after the attempt cap still returns cap. A success status beats a body that says slow down.
Fourteen rows that must stay put
Fourteen rows cover the branches a mover is likely to touch. Each row records inputs and the full return dict. If a row fails, stop and do not extract code.
| id | attempt | status | elapsed_ms | body | retry | delay_ms | reason |
|---|---|---|---|---|---|---|---|
| r01 | 5 | 500 | 0 | (empty) | false | 0 | cap |
| r02 | 8 | 429 | 10 | slow down | false | 0 | cap |
| r03 | 1 | 201 | 5 | slow down | false | 0 | ok |
| r04 | 1 | 429 | 0 | (empty) | true | 2000 | rate |
| r05 | 2 | 429 | 100 | (empty) | true | 4000 | rate |
| r06 | 3 | 429 | 30001 | (empty) | true | 2000 | rate |
| r07 | 1 | 418 | 0 | Please Slow Down | true | 2000 | rate |
| r08 | 1 | 500 | 0 | (empty) | true | 400 | server |
| r09 | 2 | 503 | 10 | (empty) | true | 500 | server |
| r10 | 3 | 404 | 0 | upstream timeout | true | 750 | server |
| r11 | 2 | 409 | 1000 | (empty) | true | 750 | conflict |
| r12 | 2 | 409 | 5000 | (empty) | false | 0 | drop |
| r13 | 4 | 404 | 0 | missing | false | 0 | drop |
| r14 | 0 | 502 | 0 | (empty) | true | 0 | server |
An empty body cell means an empty string, not a null skip. Rows one through three lock the cap and success exits. Rows four through seven lock rate-limit delay and the clamp.
Rows eight through eleven lock server and conflict paths. Rows twelve through fourteen lock the remaining drop cases. Reason strings are part of the contract in this pass.
Rename them only after a separate, reviewed change. Partial asserts hide the exact field you meant to protect. Row r14 freezes a zero delay with retry still true.
Do not clean that row during the extract. The zero delay is a frozen oddity, not a typo. A product owner must approve any later repair.
Numbered move
1. Pin the function
Copy decide_retry into the test module without edits. Keep the original file import path stable for now. A move of files is a second change, not this one.
2. Store rows as data
Put the fourteen cases in one tuple named CASES. Data rows are easier to diff than scattered asserts. Add a case id so failures name the broken row.
3. Assert the whole dict
Compare the actual dict to the expected dict. Include retry, delay_ms, and reason in every row. Do not print and eyeball the result as proof.
4. Run before any extract
Run the contract file with the standard library runner. Require fourteen passes before you open the helper. A failing row is a stop, not a prompt to rewrite.
python -m unittest test_retry_contract.py
The command above is the local check for this sample. It uses unittest from the Python standard library. No third-party plugin is required for the contract.
5. Extract one helper only
Move delay math into backoff_ms and nothing else. Pass attempt, status, elapsed_ms, and body as arguments. Leave reason strings inside decide_retry for this commit.
def backoff_ms(attempt, status, elapsed_ms, body):
text = (body or '').lower()
if status == 429 or 'slow down' in text:
delay = 1000 * (2 ** attempt)
if elapsed_ms > 30000:
return min(delay, 2000)
return delay
if 500 <= status <= 599 or 'timeout' in text:
if attempt == 1:
return 400
return 250 * attempt
if status in (408, 409) and elapsed_ms < 5000:
return 750
return 0
def decide_retry(attempt, status, elapsed_ms, body):
text = (body or '').lower()
if attempt >= 5:
return {'retry': False, 'delay_ms': 0, 'reason': 'cap'}
if status in (200, 201, 204):
return {'retry': False, 'delay_ms': 0, 'reason': 'ok'}
delay = backoff_ms(attempt, status, elapsed_ms, body)
if status == 429 or 'slow down' in text:
return {'retry': True, 'delay_ms': delay, 'reason': 'rate'}
if 500 <= status <= 599 or 'timeout' in text:
return {'retry': True, 'delay_ms': delay, 'reason': 'server'}
if status in (408, 409) and elapsed_ms < 5000:
return {'retry': True, 'delay_ms': delay, 'reason': 'conflict'}
return {'retry': False, 'delay_ms': 0, 'reason': 'drop'}
The helper must return an int delay, not a dict. That narrow type makes the extract easy to review. Callers outside the module should not import it yet.
The public return shape stays a three-key dict. Branch order in decide_retry stays identical to the original. Only the arithmetic moves into the new helper.
6. Rerun the same rows
Run the same fourteen rows after the extract. A green run means behavior stayed put, not that it is correct. Commit the helper only when every case id still matches.
Contract file you can copy
The file below is an unexecuted proposal for review. It binds the original function to the fourteen rows. Swap in the helper only after the first run is green.
import unittest
def decide_retry(attempt, status, elapsed_ms, body):
text = (body or '').lower()
if attempt >= 5:
return {'retry': False, 'delay_ms': 0, 'reason': 'cap'}
if status in (200, 201, 204):
return {'retry': False, 'delay_ms': 0, 'reason': 'ok'}
if status == 429 or 'slow down' in text:
delay = 1000 * (2 ** attempt)
if elapsed_ms > 30000:
delay = min(delay, 2000)
return {'retry': True, 'delay_ms': delay, 'reason': 'rate'}
if 500 <= status <= 599 or 'timeout' in text:
delay = 250 * attempt
if attempt == 1:
delay = 400
return {'retry': True, 'delay_ms': delay, 'reason': 'server'}
if status in (408, 409) and elapsed_ms < 5000:
return {'retry': True, 'delay_ms': 750, 'reason': 'conflict'}
return {'retry': False, 'delay_ms': 0, 'reason': 'drop'}
CASES = (
('r01', 5, 500, 0, '', False, 0, 'cap'),
('r02', 8, 429, 10, 'slow down', False, 0, 'cap'),
('r03', 1, 201, 5, 'slow down', False, 0, 'ok'),
('r04', 1, 429, 0, '', True, 2000, 'rate'),
('r05', 2, 429, 100, '', True, 4000, 'rate'),
('r06', 3, 429, 30001, '', True, 2000, 'rate'),
('r07', 1, 418, 0, 'Please Slow Down', True, 2000, 'rate'),
('r08', 1, 500, 0, '', True, 400, 'server'),
('r09', 2, 503, 10, '', True, 500, 'server'),
('r10', 3, 404, 0, 'upstream timeout', True, 750, 'server'),
('r11', 2, 409, 1000, '', True, 750, 'conflict'),
('r12', 2, 409, 5000, '', False, 0, 'drop'),
('r13', 4, 404, 0, 'missing', False, 0, 'drop'),
('r14', 0, 502, 0, '', True, 0, 'server'),
)
class RetryContractTest(unittest.TestCase):
def test_rows_hold(self):
for row in CASES:
case_id, attempt, status, elapsed_ms, body, retry, delay_ms, reason = row
with self.subTest(case_id=case_id):
actual = decide_retry(attempt, status, elapsed_ms, body)
expected = {
'retry': retry,
'delay_ms': delay_ms,
'reason': reason,
}
self.assertEqual(actual, expected)
if __name__ == '__main__':
unittest.main()
Save that listing as test_retry_contract.py before the helper edit. The subTest label should print the failing case id. A count other than fourteen means you stop.
What may move, and what must wait
Use the table below as a gate, not as style advice. One yes row is the maximum for the first commit. Two yes rows mean you are rewriting, not extracting.
| Change | First commit | Gate |
|---|---|---|
| Move delay arithmetic into backoff_ms | Yes | delay_ms column already frozen |
| Rename reason strings | No | separate commit after a log check |
| Reorder branch checks | No | order is frozen by r02 and r03 |
| Repair the zero-delay retry in r14 | No | needs a product note and a new row |
| Move the module into a new package | No | import path is out of scope |
| Retry plain 404 responses | No | that is a behavior change |
Read the gate before you open the editor. A yes row still needs the same fourteen asserts. A no row waits even if the diff looks small.
Where a free model and a free server fit
MonkeyCode's free model access can draft extra boundary rows from this table. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Review every drafted row against the current function before you keep it.
A free server option can run the contract file off production hosts. Use that isolation when the repo still imports live clients. Neither option is a quota, hardware, or uptime promise.
Ask the model for rows, not for a full rewrite. Paste the function and the existing fourteen cases as context. Reject any suggestion that changes a frozen delay.
The server's job is execution, not judgment. You still read the diff and accept the helper. A green remote run does not replace that review.
Failure patterns to check
A common failure is asserting only the retry boolean. The delay can change while that boolean stays true. Whole-dict asserts catch that slip on the next run.
Another failure is extracting two helpers in one commit. Blame becomes unclear when both extracts land together. Split them so the contract run has one suspect.
A third failure is calling the network inside the test. Characterization tests should stay pure, local, and fast. If a row needs a socket, the function is not ready.
Body sniffing is case-sensitive only after lowercasing. A drafted row that skips lowercasing will look like a bug. Compare drafted inputs with the function before you trust them.
Duplicated branch checks are an expected scar of this extract. Remove that duplication only in a later commit. The first commit should stay narrow and reversible.
Limitations
Characterization rows also freeze known bugs on purpose. A later product fix needs a new row and a written reason. These tests do not prove the delay is fair or safe.
Fourteen rows are a sample, not full branch coverage. Empty bodies, huge attempts, and non-integer status still sit outside. Add those rows only when you are about to touch them.
Free model drafts can invent statuses the function never sees. Free server runs can fail for account or network reasons. Keep a local unittest run as the source of record.
This article does not claim measured speed or cost savings. No model name, quota, or hardware figure is supplied here. Treat availability claims as current only if your account shows them.
Who should skip this approach
Skip this method when the function sends mail or charges cards. A frozen bug in that path can repeat real harm. Fix the side effect boundary before any characterization pass.
Skip it when a security flaw must not be preserved. Auth bypasses and signature skips are not useful contracts. Write a failing security test and patch that path first.
Skip it when tests can only run against production data. Characterization fixtures should be synthetic and checked in. Live customer payloads do not belong in this file.
Skip it when the team cannot review a one-helper diff. The method fails if the next step is always a rewrite. Use a narrower module, or do not start the extract.
Close the loop
The conclusion stays small on purpose in this plan. Freeze fourteen retry outcomes, then move backoff math. Leave reason strings, file paths, and policies for later commits.
If your account already shows a free server, run the file there first. That single run is the only product step this plan needs. Local green rows remain the gate you merge on.
Top comments (0)