DEV Community: Dakota Huang

The Smallest Safe Change: A Characterization-First Workflow for Messy Repos

Dakota Huang — Mon, 31 Aug 2026 00:51:22 +0000

Changing legacy code feels like defusing a bomb. One line moved, and a hidden side-effect explodes. The fix isn't courage. It's characterization tests.

A characterization test locks current behavior. It doesn't judge whether the behavior is correct. It just records what the code actually does. Once you have that lock, the smallest safe change becomes a routine edit.

This workflow uses free tooling. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model tier to draft test skeletons, and its free server option as a disposable test runner. Both are optional. The core method works with any editor and any local runner.

Step 1: Pick one function, not a whole service

Resist the urge to refactor an entire module. Choose a single function with visible inputs and outputs. Prefer one with no database calls and no randomness. That keeps your characterization test deterministic.

Here's a legacy endpoint I often use as a teaching example:

// app.js — legacy Express route, do not "fix" yet
app.get('/discount', (req, res) => {
  const price = Number(req.query.price) || 0;
  const vip = req.query.vip === 'true';
  let discount = 0;
  if (price > 100) {
    discount = 0.1;
  }
  if (vip) {
    discount += 0.05;
  }
  res.json({ final: price * (1 - discount), discount });
});

It looks simple. It has an edge-case bug waiting for you. price = -50 still gets a 10% discount. That's the behavior you must lock before changing anything.

Step 2: Draft the characterization test with a free model

Writing a test from scratch is tedious. A free language model can draft a skeleton fast. But you are the reviewer. The model does not know your repo's conventions.

I pasted the route above into MonkeyCode's free model tier with this prompt:

Write a Node.js test using node:test and assert for this Express route. Capture current behavior for prices 0, 50, 150, and vip true/false. Do not fix anything.

It generated something close to this:

// test/discount.test.js
const test = require('node:test');
const assert = require('node:assert');
const app = require('../app');

function get(path) {
  return app.handle({ method: 'GET', url: path, headers: {} }, {});
}

test('price 150, not vip', () => {
  const res = get('/discount?price=150');
  assert.deepStrictEqual(res.body, { final: 135, discount: 0.1 });
});

That draft is wrong in two ways. It expects a real HTTP server, not a handler mock. And it ignores the price=0 case. Do not copy it verbatim.

Step 3: Human-verify every assertion before running

This is the critical step. A characterization test that encodes your assumption is just a unit test with extra steps. You need the actual current output.

Run the server manually first:

node app.js &
curl 'http://localhost:3000/discount?price=150'
# -> {"final":135,"discount":0.1}
curl 'http://localhost:3000/discount?price=0'
# -> {"final":0,"discount":0}
curl 'http://localhost:3000/discount?price=-50'
# -> {"final":-45,"discount":0.1}   <-- surprising but current

Now rewrite the model's draft with the observations you just made:

// test/discount.test.js
const test = require('node:test');
const assert = require('node:assert');
const request = require('supertest');
const app = require('../app');

test('locks current discount behavior', async () => {
  const cases = [
    ['/discount?price=0', { final: 0, discount: 0 }],
    ['/discount?price=50', { final: 50, discount: 0 }],
    ['/discount?price=150', { final: 135, discount: 0.1 }],
    ['/discount?price=-50', { final: -45, discount: 0.1 }],
    ['/discount?price=150&vip=true', { final: 127.5, discount: 0.15 }],
  ];
  for (const [url, expected] of cases) {
    const res = await request(app).get(url);
    assert.deepStrictEqual(res.body, expected);
  }
});

Run it. It must pass. If it fails, your expectation is wrong. Fix the test to match reality, not the other way around.

Step 4: Use the free server as a throwaway runner

You do not need a persistent CI pipeline for one repo. MonkeyCode's free server option gives you a disposable environment. Push a branch, run the test suite there, and destroy the machine.

The workflow is boring by design:

git checkout -b characterization/discount
# add test file
npm test

When the suite passes, you have a green lock. The refactor can start.

Step 5: Make the smallest safe change and re-run

Now change one line. Fix the negative-price edge case, for example:

if (price > 100 && price > 0) {
  discount = 0.1;
}

Re-run the characterization test. One case fails: -50 now returns { final: -50, discount: 0 } instead of { final: -45, discount: 0.1 }.

That failure is a gift. It tells you exactly which behavior you intentionally changed. Update the test to reflect the new desired behavior:

['/discount?price=-50', { final: -50, discount: 0 }],

Run again. Green. Commit with a message that names both the change and the test update.

When this workflow is the wrong choice

Do not use characterization tests for greenfield code. They lock bugs as much as features. And if the legacy function has external side effects — writes to a database, sends emails, mutates shared state — you need dependency injection first. Characterization tests shine only when behavior is observable through pure inputs and outputs.

Also: never trust the AI-generated test without running it. I saw the model skip negative prices and mis-write the response mock. The value of the free model is a starting point, not a verdict.

The realistic payoff

A one-line refactor with characterization coverage takes about thirty minutes. The test suite stays after the fix. It becomes regression armor for the next engineer who touches that route.

You do not need a powerful machine or a paid CI plan. A free model to draft, a free server to run, and a human who verifies — that's the whole stack. The smallest safe change is the one you can measure before and after.

That's the entire discipline. Lock. Verify. Change. Re-lock.

From Characterization Tests to Minimal Safe Refactors: A Free-Resource Workflow

Dakota Huang — Sun, 30 Aug 2026 21:48:30 +0000

Legacy code resists change. You know it by the fear in every PR review. The fix is not more courage. It is a measurable behavior baseline, and a rule that says: change nothing until the baseline exists.

This workflow uses free resources end-to-end. No new stack. No big upfront investment. You write characterization tests, run them against a free server, and make the smallest safe edit that keeps the fingerprint identical.

Why characterization tests first

Characterization tests lock current behavior. They do not assert what you wish the code did. They assert what it actually does, bugs and all.

Identification, not correction, is the goal. A characterization test tells you when behavior changes, not whether the change is good.

For a messy repo, this is gold. You get a red/green signal before you touch the logic.

Step 1: Choose the module and record its boundary

Pick a high-fan-in, low-quality module. Internal utilities and public handlers work best. Steer clear of UI glue and hard-to-isolate side effects.

Write a short boundary list:

- function: apply_discount(cart, code)
- inputs: cart dict, code string, optional user tier
- outputs: new total, applied discount, error list
- side effects: none

This list becomes your test plan.

Step 2: Generate test candidates with a free model

Copy that boundary list into a free coding model, for example the one bundled inside MonkeyCode's free tier. Ask for brute-force input/output pairs only, not assertions about intended behavior.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Example request:

Generate 20 raw test cases for apply_discount. Do not judge correctness. Just cover empty cart, duplicated codes, unavailable codes, negative quantities, and mixed item types.

The model returns candidates. You do not trust them. You verify them by running.

Step 3: Run a baseline on a free server

Free server options are common now. I used a free instance for this run. The point is to get execution results without paying for your laptop's uptime.

Set up a tiny runner:

# fingerprint.py
import hashlib, json
import your_module

def capture(func, case):
    try:
        out = func(**case["inputs"])
        return {"ok": True, "out": out}
    except Exception as exc:
        return {"ok": False, "out": str(exc)}

def fingerprint(cases):
    lines = []
    for c in cases:
        res = capture(your_module.apply_discount, c)
        digest = hashlib.sha256(json.dumps(res, sort_keys=True).encode()).hexdigest()
        lines.append(digest)
    return "\n".join(lines)

Run it once and save the output as baseline.txt. This file is your behavior fingerprint.

Step 4: Prove the tests catch change

A baseline that never moves is useless. You need to know your tests detect refactors.

Introduce a deliberate mutation. For example, reverse the discount eligibility condition. Run the fingerprint again.

python fingerprint.py > current.txt
diff baseline.txt current.txt

You should see mismatches. If you do not, your test cases are too weak. Add more until mutations change the fingerprint.

Rule of thumb: three distinct mutations, three changed fingerprints. Otherwise, the suite is not safe to drive refactoring.

Step 5: The minimal safe change loop

Now the actual refactor works in tiny steps:

Reorder one internal helper. Run the fingerprint.
Extract a repeated block. Run the fingerprint.
Rename variables. Run the fingerprint.
Touch one behavior only. Run the fingerprint and inspect the diff.

The order matters. Structural changes first, behavioral changes last.

What the workflow protects

Silent logic flips get caught immediately.
Refactors stay reviewable because each diff is small.
The team gains a permanent regression net after the dust settles.

Limitations and when not to use this

This approach does not fix design. It fixes safety.

It cannot detect conjunctions of multiple changes.
It only covers exercised input space. Blind spots remain.
It will lock in bugs. If the code is wrong today, your baseline says wrong is correct.

Do not use it for greenfield code with clear contracts. Write real unit tests there.

Do not use it when the module has huge external side effects. Your fingerprint will not capture database writes or network calls cleanly.

And do not expect a free server to handle expensive integration suites. Keep the baseline small and focused.

The takeaway

Characterization tests give you the floor. Minimal safe changes keep you from falling through it. Free models generate the candidates, and a free server runs the baseline. The whole loop costs you time, not money.

Try it on your messiest module this week. Start with the boundary list, not the refactor.

Characterize, Minify, Refactor: A Safe Path Through a Messy Codebase

Dakota Huang — Sun, 30 Aug 2026 18:46:39 +0000

When you refactor code with no tests, you are flying blind.
Characterization tests give you a map.
Write them first. Change code in tiny steps. Verify after each step.

This post shows a concrete workflow.
You will see a messy discount function.
You will lock its behavior with characterization tests.
Then you will refactor it without breaking a single case.

The Messy Function

Here is a legacy function I found in a payment service.
It calculates discounts. No one knows all the rules.

def apply_discount(price, customer, code, items_count):
    final = price
    if customer == "vip":
        final = price * 0.9
    elif code == "SUMMER":
        final = price * 0.8
    if items_count >= 5 and customer != "basic":
        final = final - 2.0
    if final > 200:
        final = final * 0.95
    if customer == "basic" and code == "SUMMER":
        final = final + 5.0
    if final < 0:
        final = 0
    return round(final, 2)

Nobody can explain why SUMMER adds five dollars for basic customers.
It is probably a bug. But it is shipped behavior.

Step 1: Characterize Before You Change

Characterization tests record what the code does today.
They do not test what should happen.
They test what does happen.

Write one test per input dimension.
Use representative values: boundaries, nulls, unusual combinations.

import pytest

def test_apply_discount_regular():
    assert apply_discount(100, "regular", "", 1) == 100

def test_apply_discount_vip():
    assert apply_discount(100, "vip", "", 1) == 90

def test_apply_discount_summer():
    assert apply_discount(100, "regular", "SUMMER", 1) == 80

def test_apply_discount_basic_summer_bug():
    assert apply_discount(100, "basic", "SUMMER", 1) == 85  # locked bug

def test_apply_discount_items_discount():
    assert apply_discount(100, "regular", "", 5) == 98

Run them. Watch them pass.
Now every future change is measurable.

Step 2: Generate Draft Tests, Then Verify

You can ask an AI model to suggest test cases.
I used MonkeyCode's free model access for this.
The suggestions were useful as a checklist.
They were not correct enough to copy blindly.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The model suggested edge cases like negative prices and zero items.
Those were good additions.
It also suggested a test for code being None, which the code would crash on.
That is a real gap, but adding that test first would break the suite.
Characterization tests should match current behavior, not ideal behavior.
So I added only the cases that passed.

def test_apply_discount_negative_price():
    # Current behavior: negative price becomes 0 after checks
    assert apply_discount(-50, "regular", "", 1) == 0

Use AI drafts as a starting point.
Trust your own verification over the model's confidence.

Step 3: Create a Baseline on a Free Server

A test suite must run somewhere.
You can run it locally, but team members need a shared baseline.
MonkeyCode's free server option gives you a temporary remote runner.
Push a branch. The server runs your characterization suite.

git checkout -b characterize-discount
# add tests to test_discount.py
git push origin characterize-discount
# run the suite on the free server
monkeycode ci --file test_discount.py

The output is a clear pass/fail list.
Save that log as your golden baseline.
Every refactor commit will be compared against it.

Step 4: Refactor in Smallest Steps

Do not rewrite the whole function at once.
Extract one rule at a time. Run tests after every commit.

Start with the simplest extraction: the customer discount.

def _customer_discount(price, customer):
    if customer == "vip":
        return price * 0.9
    return price

Change apply_discount to use it.

def apply_discount(price, customer, code, items_count):
    final = _customer_discount(price, customer)
    if code == "SUMMER":
        final = final * 0.8
    if items_count >= 5 and customer != "basic":
        final = final - 2.0
    if final > 200:
        final = final * 0.95
    if customer == "basic" and code == "SUMMER":
        final = final + 5.0
    if final < 0:
        final = 0
    return round(final, 2)

Run the tests. They pass.
The behavior is identical.
Commit with a clear message.

git commit -m "Extract VIP discount rule"

Repeat for each rule.
After four extractions, the function reads clearly.

def apply_discount(price, customer, code, items_count):
    final = _base_discount(price, customer)
    final = _apply_code_discount(final, code)
    final = _apply_volume_discount(final, customer, items_count)
    final = _apply_cap_discount(final)
    final = _apply_basic_summer_penalty(final, customer, code)
    return _round_non_negative(final)

Step 5: Diff Before and After

A characterization suite ensures each step keeps old behavior.
But a missing test can hide a regression.
Use a differential check on random inputs.

Generate random tuples of the four arguments.
Run the old and new functions on each.
Compare outputs.

import random

def diff_check(old_func, new_func, samples=1000):
    for _ in range(samples):
        price = random.randint(-100, 2000)
        customer = random.choice(["regular", "vip", "basic", "none"])
        code = random.choice(["", "SUMMER", "WINTER", "VIP"])
        items = random.randint(0, 10)
        old = old_func(price, customer, code, items)
        new = new_func(price, customer, code, items)
        if old != new:
            return price, customer, code, items, old, new
    return None

Run it before merging.
Any mismatch means you missed a rule.

Limitations

Characterization tests do not verify correctness.
They verify consistency.
If the original code has a bug, the test locks the bug in.

This workflow suits risk-averse refactors.
It slows you down. That is the point.
Do not use it for greenfield code or for features you are actively changing.
Do not blindly trust AI-generated test drafts.
You are responsible for the safety net.

The Pattern

Characterize. Minify. Refactor. Verify.
That is the cadence.

Start with one messy function.
Lock its behavior with tests.
Refactor in tiny commits.
Diff before you merge.

The smallest safe change is one you can undo in seconds.
Make every commit that small.

You're the Reviewer Now. Who Reviews Your Verdict?

Dakota Huang — Sun, 30 Aug 2026 15:44:07 +0000

AI turned every developer into a reviewer. Nobody validates the reviewer. When an AI suggests a refactor, your approval is the gate. Without a behavior baseline, your verdict is a guess.

This post explains how to test your own review. It uses characterization tests to lock current behavior. Then it measures an AI diff against that lock. The method is repo-agnostic and costs almost nothing.

The reviewer's blind spot

You see a diff. The AI calls it a refactor. The test suite is green. You approve. Weeks later, a regression appears.

The tests were written for the old shape. They rarely cover the edge cases a refactor touches. Your review needs a second signal.

Characterization tests provide that signal. They record what the code does today, not what it should do. They turn implicit behavior into executable assertions.

Step 1: Record the behavior fingerprint

Take a legacy function. Here is a typical price parser:

# legacy.py
def parse_price(value):
    if value is None:
        return 0
    if isinstance(value, str):
        return float(value.replace("$", "").replace(",", ""))
    return float(value)

Write a characterization test for the input shapes you know exist:

import unittest
from legacy import parse_price

class TestParsePrice(unittest.TestCase):
    def test_existing_behaviors(self):
        cases = [
            ("$1,234.50", 1234.5),
            (None, 0),
            (" 42 ", 42.0),
            ("0", 0.0),
            ("-7", -7.0),
        ]
        for raw, expected in cases:
            with self.subTest(raw=raw):
                self.assertEqual(parse_price(raw), expected)

Run it on the legacy code. It must pass. If it fails, your test does not match production behavior. Fix the test, not the code.

Step 2: Draft edge cases with a free model

Five cases are not enough. You need more input families. A free model can propose candidates quickly.

For example, MonkeyCode's free model access can draft a table of edge cases in seconds.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat that draft as raw material, not truth. Verify each candidate against the legacy behavior. This is the same rule you apply to the refactor. AI output never skips the verification step.

Step 3: Diff the implementations

Now you receive a proposed refactor. A common simplification is:

# new_impl.py
def parse_price(value):
    return 0 if value is None else float(str(value).replace("$", "").replace(",", ""))

Run your characterization tests against both versions. Then run a differential check. Use a generator that draws from real call-site patterns, not pure randomness:

import random

def realistic_input():
    r = random.random()
    if r < 0.1:
        return None
    if r < 0.6:
        return f"${random.randint(0, 100000):,}"
    return random.uniform(-1000, 1000)

for _ in range(2000):
    raw = realistic_input()
    old = parse_price_old(raw)
    new = parse_price_new(raw)
    if old != new:
        print("divergence:", repr(raw), old, new)
        break

If you want to run this loop without touching your production environment, MonkeyCode's free server option is a low-friction place to do it. Spin up a scratch environment, paste both functions, run the generator. The result is a pass or a divergence list.

Step 4: Make a verdict

Use a decision table. This is the core of reviewing the reviewer.

Signal	Verdict
Characterization tests pass on both versions	Proceed to manual diff review
Differential test finds a divergence	Reject; ask AI to preserve old behavior
Diff touches a function with fan-in > 5	Reject unless you add caller tests
No characterization baseline exists	Reject before reading the diff

The last row is the most common failure. Teams approve AI refactors because tests are green. The tests were too weak to notice the change.

Limitations

This workflow protects only observable behavior. It misses concurrency, timing, and external side effects. It also struggles when the current behavior is a bug you want to fix. In that case, write a regression test for the new behavior first.

Who should skip this? Solo developers on throwaway prototypes. If the code dies in a week, characterization overhead is waste. Use it when the code has real callers.

The honest verdict

Your review is as strong as the behavior lock beneath it. Free model access and a free server make the loop cheap, but they do not replace judgment.

Next time you accept an AI refactor, ask one question: "What did I lock before I approved?" If the answer is nothing, you did not review. You guessed.

The Smallest Safe Change: Capture Behavior, Refactor, Ship

Dakota Huang — Sun, 30 Aug 2026 12:40:49 +0000

The smallest safe change is the one you can revert without pain. Characterization tests make that possible. This workflow turns a legacy function into a locked baseline.

The Messy Function

Legacy code is a black box. You cannot trust your reading. You can trust captured behavior.

Here is a typical function. It reads global state. It has hidden fees. Zero tests.

# shipping.py
CONFIG = {"discount": 0.9, "holiday": False}

def shipping_cost(order, customer):
    base = 5.0
    if order["weight"] > 10:
        base += 10.0
    if order.get("expedited"):
        base += 15.0
    if customer["tier"] == "gold":
        base *= CONFIG["discount"]
    if CONFIG["holiday"]:
        base *= 0.95
    return round(base, 2)

You can read it, but you cannot trust your reading. The function depends on a mutable global. One wrong assumption breaks production.

The strategy: capture what it does today. Then change one line and verify.

Step 1: Choose a Capture Source

You need realistic inputs. Use production logs, past test data, or a hand-picked set of edge cases.

Source	Pros	Cons
Production logs	Covers real traffic	Needs parsing
Manual examples	Fast, targeted	Misses unknown branches
Property generation	Explores many combos	Over-generates noise

For this example, manual examples are enough. Copy them from your ticket descriptions or your reviewer's brain.

Step 2: Instrument and Capture

Wrap the function with a decorator. Every call gets recorded as JSON.

# instrument.py
import json
import functools
from shipping import shipping_cost

def characterize(func, log="capture.jsonl"):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        record = {"args": args, "kwargs": kwargs, "result": result}
        with open(log, "a") as f:
            f.write(json.dumps(record) + "\n")
        return result
    return wrapper

characterized = characterize(shipping_cost)

# Feed realistic inputs
characterized({"weight": 5}, {"tier": "regular"})
characterized({"weight": 15, "expedited": True}, {"tier": "gold"})
characterized({"weight": 11}, {"tier": "silver"})

Open capture.jsonl. You see three records with exact outcomes. That is your evidence.

Expand the capture set if needed. Add more combinations. Capture the global state too if it changes.

Step 3: Generate Tests From Captures

Do not hand-write tests. Generate them from the captured records.

# generate_tests.py
import json

records = [json.loads(line) for line in open("capture.jsonl")]

with open("test_shipping.py", "w") as f:
    f.write("import unittest\nfrom shipping import shipping_cost\n\n")
    f.write("class TestShipping(unittest.TestCase):\n")
    for i, rec in enumerate(records):
        f.write(f"    def test_case_{i}(self):\n")
        f.write(f"        self.assertEqual(shipping_cost(*{rec['args']!r}, **{rec['kwargs']!r}), {rec['result']!r})\n")
    f.write("\nif __name__ == '__main__':\n    unittest.main()\n")

Run it.

python -m unittest test_shipping

If your inputs are representative, you get green. This baseline is your safety net.

Step 4: Lock and Refactor

Now make the smallest possible change. Move the expedited fee into CONFIG. Replace the hardcoded 15.0 with a lookup.

# shipping.py (after change)
CONFIG = {"discount": 0.9, "holiday": False, "expedited_fee": 15.0}

def shipping_cost(order, customer):
    base = 5.0
    if order["weight"] > 10:
        base += 10.0
    if order.get("expedited"):
        base += CONFIG["expedited_fee"]
    if customer["tier"] == "gold":
        base *= CONFIG["discount"]
    if CONFIG["holiday"]:
        base *= 0.95
    return round(base, 2)

Re-run the tests.

python -m unittest test_shipping

Green again. You shipped a safe refactor. Behavior is identical. You only changed the source of a constant.

The Bug You Must Not Fix

You will find bugs while capturing behavior. Ignore them for now. Your job is to preserve behavior, not improve it. Ship the refactor first. Then open a separate ticket for the bug.

If you fix the bug inside the same change, you cannot tell which change broke something. Keep refactor and fix separate. This is non-negotiable.

Where an AI Assistant Helps

This workflow stands alone. But free tooling can compress it. MonkeyCode offers free model access and a free server option. You can use them to draft generator scripts or run the capture in an isolated environment. It is optional.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Limitations

Characterization tests only cover observed inputs. Hidden branches stay untested. If production traffic hits an edge case you never captured, your next refactor can still break it.

Nondeterministic behavior is a trap. If your function depends on time or randomness, capture those values explicitly. Otherwise your tests will fail on replay.

Global state is another trap. If CONFIG changes at runtime, your captured inputs are incomplete. Record the state alongside the arguments, or freeze it before capture.

Who Should Not Use This Workflow

Skip this approach in three cases. You already have a contract or spec. Your codebase has comprehensive unit tests. Or your legacy function is badly broken and you actually need to fix behavior, not preserve it.

If you are adding a brand-new feature, write normal tests. Characterization tests are for code you fear to touch.

Conclusion

Characterization-first refactoring is simple. Capture behavior, lock it with tests, then change one line. The smallest safe change is the one you can revert without pain.

Pick one ugly function today. Capture five inputs. Generate tests. Refactor one constant. Your future self will thank you.

Characterization Tests Need a Home: Run Them on a Free Server

Dakota Huang — Sun, 30 Aug 2026 11:35:15 +0000

Characterization tests turn messy code into a refactor target. They record current behavior before you change anything. Then you can refactor with confidence.

Here's the workflow I use for those scary legacy functions. It runs on a free server, so you get a baseline without standing up a CI machine.

The Problem: Refactoring Without Tests

Messy code has hidden behavior. You see one branch, but there are edge cases in the shadows.

You change a line. A feature breaks. Nobody knows why.

The solution is simple: freeze the behavior first. Write tests that assert what the code actually does, not what it should do.

Step 1: Pick the Messiest Function

Choose a function with many branches, side effects, or copied logic. For this example, let's use a price calculator.

def calculate_price(item_type, quantity, discount_code):
    base = 10 if item_type == "book" else 20
    if quantity > 5:
        base = base * 0.9
    if discount_code == "SAVE10":
        base = base * 0.9
    return round(base * quantity, 2)

This function has two magic numbers and two hidden rules. Perfect for characterization.

The goal is to capture every input–output pair you can think of.

Step 2: Write a Behavior Capture Harness

Create a script that feeds a set of inputs to the function and logs the outputs. This harness is your oracle.

import json

cases = [
    ("book", 3, ""),
    ("book", 6, ""),
    ("book", 6, "SAVE10"),
    ("game", 1, ""),
    ("game", 10, "SAVE10"),
]

results = {}
for item, qty, code in cases:
    key = f"{item}:{qty}:{code}"
    results[key] = calculate_price(item, qty, code)

with open("baseline.json", "w") as f:
    json.dump(results, f, indent=2)

print("Baseline captured")

Run this script before you touch anything. Save the output as baseline.json.

That file is your contract.

Step 3: Move the Harness to a Free Server

You don't need to run this locally. You can host it on a free server and run it on a schedule.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server tier. The free tier includes 10 million tokens for AI usage and a small always-on server. That's enough to run a cron job that re-captures behavior daily.

Here's a simple Dockerfile for your harness:

FROM python:3.11-slim
WORKDIR /app
COPY harness.py .
CMD ["python", "harness.py"]

Deploy this container to MonkeyCode's free server. Add a cron rule to run it every morning.

Now you have a living baseline. If someone changes the function, the next run detects the difference.

Step 4: Make the Smallest Safe Change

Now you refactor. Keep the change atomic. Extract a helper, rename a variable, or simplify a condition.

For our function, we can extract the discount logic:

def apply_discount(price, code):
    if code == "SAVE10":
        return price * 0.9
    return price

def calculate_price(item_type, quantity, discount_code):
    base = 10 if item_type == "book" else 20
    if quantity > 5:
        base = base * 0.9
    return round(apply_discount(base, discount_code) * quantity, 2)

Run the harness again. Compare the new baseline.json with the old one.

If the outputs match, your refactor is behavior-preserving. If they differ, you've found a behavior change before it reaches production.

Step 5: Automate the Comparison

Don't compare manually. Write a simple diff script:

python harness.py
python -c "
import json
old = json.load(open('baseline.json'))
new = json.load(open('current.json'))
print('Match' if old == new else 'Mismatch')
"

On your free server, a failing diff can email you or post to a Slack webhook.

Now every deployment candidate gets checked against the frozen behavior.

Limitations and Who Should Skip This

This workflow assumes you have a function you can call directly. If the code is deeply coupled to network I/O or a database, you need a different harness.

It also only covers the inputs you thought to test. Weird edge cases will still hide.

Characterization tests are not a replacement for real unit tests. They are a safety net for legacy code you don't understand yet.

Skip this if you already have good coverage. Skip it if the code is trivial and safe. Use it when the code is messy and scary.

The Takeaway

Characterization tests let you refactor with facts, not vibes.

A free server makes the baseline continuous. You don't have to babysit a local script.

Try this flow on your worst function. You'll learn more about it in an hour than in a week of reading.

Real API Traffic Is Your Missing Test Data: A Free-Server Capture Workflow

Dakota Huang — Sun, 30 Aug 2026 06:36:00 +0000

Real API traffic is the hardest test data to get and the most valuable one to have. Unit tests manufactured inside a test suite are clean. Production requests are not. They contain empty strings, huge payloads, duplicated keys, unexpected encodings, and timing-sensitive fields. You know this. The problem is you cannot just install a tcpdump next to your production API and start recording.

You need a capture point outside production. It should be always-on, separate from your own machine, and cheap. A free server is exactly that. This article shows a workflow that records real requests to a staging or secondary API endpoint, then uses a free model to mine that capture for edge cases worth turning into tests. No production changes, no sensitive data hoarding, and no invented fixtures.

The Capture Proxy

The proxy is the gateway in front of the service you want to observe. Every request passes through it, is forwarded unchanged, and only a tiny canonical description of the request/response is kept. You run the proxy on a remote host that has network access to the target service.

The core is a few lines of Node.js using the http module. It parses the request, forwards it, and captures a filtered trace.

// capture-proxy.js
const http = require("node:http");

const TARGET = new URL(process.env.TARGET_URL); // e.g. http://staging-api:3000

const seenHashes = new Set();

const server = http.createServer((req, res) => {
  let raw = "";
  req.on("data", (c) => (raw += c));
  req.on("end", () => {
    const forwardReq = http.request({
      hostname: TARGET.hostname,
      port: TARGET.port,
      path: req.url,
      method: req.method,
      headers: req.headers,
    }, (forwardRes) => {
      let responseBody = "";
      forwardRes.on("data", (c) => (responseBody += c));
      forwardRes.on("end", () => {
        const trace = makeTrace(req, raw, forwardRes, responseBody);
        const key = hash(trace);
        if (!seenHashes.has(key)) {
          seenHashes.add(key);
          appendTrace(trace);
        }
        res.writeHead(forwardRes.statusCode, forwardRes.headers);
        res.end(responseBody);
      });
    });
    forwardReq.write(raw);
    forwardReq.end();
  });
});

server.listen(8080);

The trace contains only the shape of the request, not the payload itself. You want enough to reproduce the call path but not enough to leak secrets. makeTrace implements that mask.

function makeTrace(req, body, response) {
  const url = new URL(req.url, "http://localhost");
  return {
    method: req.method,
    path: url.pathname,
    query: url.search,
    status: response.statusCode,
    bodyHash: sha1(body).slice(0, 16),
    bodySize: body.length,
    headerKeys: Object.keys(req.headers)
      .filter((k) => !k.startsWith("cookie") && k !== "authorization")
      .join(","),
  };
}

We store a hash of the body, the size, and the header names. The actual body is never written to disk. Later, when you see an interesting trace, you can replay it against the staging service manually to get the real body. That is the trick: the proxy does not need to retain sensitive data for you to build a useful corpus.

Running It on a Free Server

One practical way to get an always-on capture host without opening your wallet is to use MonkeyCode's free server option. That server can call your staging API from outside your local network. You keep the proxy on that box, let it run for a week, and collect a corpus of real requests.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free model access MonkeyCode offers then helps you mine that corpus, so you do not have to read every trace by hand.

Mining Traces With a Free Model

After a few days you have a JSONL file with hundreds of trace entries. The structure looks like this:

{"method":"POST","path":"/orders","query":"?plan=pro","status":400,"bodyHash":"a3f9...","bodySize":342,"headerKeys":"content-type,user-agent,x-request-id"}

Now the model comes in. You do not need to write a parser that guesses what is interesting. You feed a small sample of traces to a free model with a strict prompt that asks it to identify which traces would be worth replaying and turning into tests.

You are given API trace summaries. For each trace, classify it as:

- HIGH value: anomalous status, unusual body size, uncommon header set
- MEDIUM value: normal-looking call that might still hit an edge case
- LOW value: routine success

Output JSON array of objects with trace index and label. Do not explain. No code output.

MonkeyCode's free model access lets you run this classification over batches again and again without counting it against a paid quota. You can paste ten traces at a time and get labels back quickly. The goal is to reset your intuition: which calls are the strange ones? The model will usually flag a trace with a 400 response and a bodySize of 2 bytes, or a GET to /health with a query string that is never used. Those are exactly the kind of input that a test suite would never generate because they look like mistakes. They are not mistakes; they are clients being sloppy, and your API needs to handle them.

Turning Interesting Traces Into Tests

Once you have a shortlist, replay each trace against the staging service manually and record the exact request body. Then write a test that pins the server's behavior for that weird input. This is not characterization testing in the "lock everything" sense; it is targeted regression protection for the edges that matter.

test("POST /orders with empty plan query returns 400", async () => {
  const res = await fetch(
    `${API_BASE}/orders?plan=pro&plan=`,
    {
      method: "POST",
      body: "{}",
      headers: { "content-type": "application/json" },
    }
  );
  assert.equal(res.status, 400);
});

You are not testing for a specific output value; you are testing that the system does not crash and returns a controlled error. That is the real insight: production traffic often exposes "does it survive" more than "does it compute correctly".

Deciding What to Keep or Discard

Use this simple table when reviewing captured traces.

Trace property	Keep?	Reason
status >= 500	Always	Failure needs a repro
status 4xx with nonstandard body	Always	Client misuse reveals missing validation
body size >> P95	Often	Large payloads can stress buffer limits
same bodyHash but different status	Often	Same body but different outcomes suggests flakiness
Routine 200s with stable size	Skip	Already covered by happy-path tests
Contains authorization header	Never	Do not store credential metadata

The proxy already strips auth headers, but your replay step must also use safe test credentials.

Limitations and Who Should Not Use This

First, a capture point works only when you have traffic flowing to the target. If your staging API gets only one call per week, this workflow gives you almost nothing. You need a real consumer base, even if it is a handful of developers.

Second, the proxy as shown is synchronous and holds the socket open while forwarding. Under high throughput, it will start dropping or timing out. This is meant for a low-traffic staging or QA environment, not for production traffic. If you need production-scale capture, use a proper message queue and a dedicated ingestion pipeline.

Third, trace summaries hide the actual content. If you want to use them for differential testing, you will need a separate mechanism to fetch the original bodies. This method biases toward presence of oddities, not content of those oddities. For content-level analysis, you need a sanitized replay service.

Fourth, and this is important: a free model may hallucinate a reason for an edge case. Use its classifications as a signal, not as a verified truth. Replay the trace yourself before writing a test.

Who should skip this? If your project has a full contract test suite with schema validation and property-based testing, you may already cover most edge cases. This workflow is for teams that rely on examples and happy paths, and for those who want a no-cost way to find out what their real clients are doing. It is also not for security-sensitive APIs where even hashes of bodies could leak information via length.

The Net Effect

Your tests are only as good as your inputs. A free remote server costs you nothing to run the capture, and a free model shrinks the reading time from three hours to thirty minutes. The artifact is small: one Node.js script, a JSONL file, and a handful of regression tests born from real-world weirdness.

Stop inventing test cases. Start recording them.

Your AI Refactor Looks Right: Verify It With a Behavior Fingerprint

Dakota Huang — Sun, 30 Aug 2026 03:32:27 +0000

Your AI refactors look plausible. Plausible is not equivalent.

A clean diff can still change behavior. A reviewer's intuition misses arithmetic-order bugs. The model's confidence hides them. You need a machine to check equivalence. Not a deeper review. A repeatable behavior fingerprint.

The failure mode

Consider a small discount function. Two conditionals. One log records every branch.

def apply_discount(order, log):
    subtotal = order['total']

    if subtotal > 1000:
        subtotal *= 0.9
        log.append(('discount', order['id']))

    if order.get('loyalty') and subtotal > 100:
        subtotal -= 10
        log.append(('loyalty', order['id']))

    return subtotal

An assistant sees a cleanup. Move the loyalty check first. Reduce the branches.

def apply_discount(order, log):
    subtotal = order['total']

    if order.get('loyalty') and subtotal > 100:
        subtotal -= 10
        log.append(('loyalty', order['id']))

    if subtotal > 1000:
        subtotal *= 0.9
        log.append(('discount', order['id']))

    return subtotal

For a loyal customer with a 1200 order, the math drifts:

Before: 1200 → 1080 → 1070. Log: discount, loyalty.
After: 1200 → 1190 → 1071. Log: loyalty, discount.

The diff looks clean. The behavior changed. Reviewers approve this daily. Code review tests plausibility, not arithmetic.

Behavior fingerprints

A behavior fingerprint records observable effects. Same function. Same probe inputs. Same side-effect ledger. Different hash means different behavior.

A golden file captures bytes. A differential test needs a second implementation. A fingerprint only needs a probe set. Run it before the refactor. Run it after. Compare what changed.

# fingerprint.py
import hashlib
import json

PROBES = [
    ('large_plain', {'id': 'A1', 'total': 1200, 'loyalty': False}),
    ('small_loyal', {'id': 'B2', 'total': 150,  'loyalty': True}),
    ('tiny_loyal',  {'id': 'C3', 'total': 90,   'loyalty': True}),
    ('large_loyal', {'id': 'D4', 'total': 1200, 'loyalty': True}),
]

def fingerprint(apply_fn, probes=PROBES):
    rows = []
    for label, order in probes:
        log = []
        try:
            value = apply_fn(order, log)
            state = 'ok'
        except Exception as exc:
            value = None
            state = type(exc).__name__
        rows.append({
            'case': label,
            'state': state,
            'value': value,
            'log': log,
        })
    raw = json.dumps(rows, sort_keys=True).encode()
    return hashlib.sha256(raw).hexdigest(), rows

The payload records three signals per case. The return value. The exception class. The log sequence. Sorting keeps the hash stable. Stable hashes make comparisons honest.

Run it against the legacy module.

python -c "from legacy import apply_discount; from fingerprint import fingerprint; print(fingerprint(apply_discount)[0])"

Save that hash. Apply the proposed refactor. Run it again.

python -c "from refactored import apply_discount; from fingerprint import fingerprint; print(fingerprint(apply_discount)[0])"

Now compare the two hashes. A match means your probes saw the same behavior. A mismatch means something moved. Diff the rows to name the exact probe. Review has a target instead of a vibe.

Building the probe matrix

Probe matrices need four input families. Normal, boundary, failure, and side-effect-sensitive.

Normal inputs prove the happy path. Boundary inputs catch threshold moves. Failure inputs catch exception-order changes. Side-effect inputs catch logging and mutation shifts. The example above uses four probes. A payment reconciler may need forty. Add probes whenever a bug escapes.

The workflow

Six steps. No ceremony.

Freeze the interface. Change the body only. Keep the signature untouched.
Build the probe matrix. Cover happy paths, boundaries, failures, and side effects.
Generate the baseline fingerprint. Save the hash with the refactor branch.
Ask a free model for a refactor draft. MonkeyCode's free model access fits this step. Paste the legacy function and the probe list.
Run the fingerprint against the draft. Compare hashes.
Inspect only mismatched probes. Decide whether each shift is a bug or an accepted change.

Step six keeps humans in charge. The hash points at the problem. Reviewers judge the trade-off.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Where MonkeyCode fits

The probe set works without MonkeyCode. The script is plain Python. Python 3 is the only dependency.

MonkeyCode adds two practical pieces. Its free model access drafts the refactor. Its free server option can host the fingerprint script. Shared hosting keeps the team on one probe set. No drift between laptops. No works on my machine fingerprints. The method does not depend on it.

Choosing the right verification

Different refactors need different evidence. Use the right tool.

Method	Answers	Cost	Blind spot
Code review	Does the diff look right?	High	Arithmetic and order changes
Golden file	Do bytes match?	Medium	Side effects outside the file
Differential test	Do two implementations agree?	Medium	Needs a second implementation
Behavior fingerprint	Does behavior match on probes?	Low	Untested inputs

Fingerprints are cheap. They fit AI-assisted cleanup best. Golden files remain better for serialization code. Differential tests remain better when both implementations survive.

Limits

A fingerprint is evidence. It is not proof.

Matching hashes only cover the probes you wrote. Unknown inputs stay outside the evidence. Nondeterministic code breaks the method. Random values and timestamps add noise. Fix seeds first. Or move randomness outside the probed function.

Who should not use this:

Teams without stable probe inputs. UI gestures and third-party APIs resist fingerprinting.
Safety-critical systems. A hash is a floor, not a ceiling. Keep formal verification.
One-off micro-changes. The setup cost may exceed the payoff.

The real gate

Code review answers one question. Does the diff look reasonable?

It never answers the second question. Does the program behave the same? A probe-based fingerprint answers that. Cheap. Repeatable. Hostile to plausible errors.

Run it before the reviewer reads anything. Your next AI refactor deserves a witness.

Characterization-First Refactoring: Lock the Mess, Then Fix It

Dakota Huang — Sun, 30 Aug 2026 00:27:13 +0000

You don't refactor a messy repo by reading it. You refactor it by locking behavior first. Characterization tests capture what the code does today. Only then can you change it safely.

I have written refactor safety articles before. This one adds a missing step: using a free model to find blind spots. The method works for any legacy function you are afraid to touch.

The Scenario

Your legacy codebase has a function named computeTotal. It handles cart pricing, discounts, and shipping. Nobody wrote tests for it. The function changed hands five times. Your task: make it readable without changing outputs.

Here is a simplified version of the mess:

function computeTotal(items, user, promo) {
  let total = 0;
  for (const it of items) total += it.price * it.qty;
  if (user.new) total = total * 0.9;
  if (promo === 'SAVE5') total = total - 5;
  if (total > 100 && items.length > 5) total = total - 10;
  return { total, shipping: total > 50 ? 0 : 9.99 };
}

The real version is longer. It has global state and dead branches. You want to extract logic. You cannot because you fear breaking a hidden rule.

Step 1 — Write Characterization Tests First

A characterization test records current behavior. It does not judge whether that behavior is correct. It freezes the system so refactoring won't introduce silent changes.

Start with a test file that calls the function with a few cases. Run it once with empty assertions, then copy the actual output into expected.

import test from 'node:test';
import assert from 'node:assert/strict';
import { computeTotal } from '../src/pricing.js';

const cases = [
  {
    name: 'empty cart',
    input: [[], { new: false, country: 'US' }, null],
    expected: { total: 0, shipping: 9.99 }
  },
  {
    name: 'single item, existing user',
    input: [[{ price: 10, qty: 1 }], { new: false }, null],
    expected: { total: 10, shipping: 9.99 }
  },
  {
    name: 'new user gets discount',
    input: [[{ price: 100, qty: 1 }], { new: true }, null],
    expected: { total: 90, shipping: 0 }
  },
  {
    name: 'promo code SAVE5',
    input: [[{ price: 20, qty: 1 }], { new: false }, 'SAVE5'],
    expected: { total: 15, shipping: 9.99 }
  }
];

for (const c of cases) {
  test(`characterize: ${c.name}`, () => {
    assert.deepEqual(computeTotal(...c.input), c.expected);
  });
}

If you already know the output, write it directly. If not, use a temporary log script to print the structure.

Step 2 — Use a Free Model to Find Blind Spots

Hard cases hide in boundaries. Empty carts, promo codes, price thresholds, and floating-point arithmetic are classic traps. A human usually misses two or three. A model can suggest more.

I used MonkeyCode's free model access to generate edge-case ideas. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The model proposed these inputs:

Negative quantity (e.g., qty: -1)
Promo code applied twice (SAVE5 and another code)
New user plus promo code together
Total exactly 100 with exactly 5 items
Floating-point prices like 19.99 * 3
items containing an object without qty

Some proposals are invalid for your domain. That is fine. Filter them. Add the useful ones to your test matrix.

// additional cases after model feedback
{
  name: 'floating point total',
  input: [[{ price: 19.99, qty: 3 }], { new: false }, null],
  expected: { total: 59.97, shipping: 9.99 }
  // note: floating point may produce 59.96999999999999
},
{
  name: 'total > 100 and items > 5',
  input: [Array(6).fill({ price: 25, qty: 1 }), { new: false }, null],
  expected: { total: 150, shipping: 0 }
}

Be careful with floating-point assertions. Use assert.deepEqual only when numbers are exact. Otherwise compare rounded values or use a tolerance helper.

Step 3 — Run and Record the Baseline

Run the test suite now. Your assertions may fail because expected values are guesses. That is normal. Replace each expected with the actual captured output.

To capture outputs quickly, write a temporary script:

const cases = [
  [[], { new: false }, null],
  [[{ price: 10, qty: 1 }], { new: false }, null],
  [[{ price: 100, qty: 1 }], { new: true }, null]
];
for (const args of cases) {
  console.log(JSON.stringify(computeTotal(...args)));
}

Copy each printed object into your test file. Now every case passes. You have a lock on the current behavior.

Run with:

node --test tests/characterize.test.js

You should see a green suite. If anything fails, your environment differs or the function has nondeterministic behavior. Investigate before going further.

Step 4 — Make the Smallest Safe Change

Now the lock is in place. Pick one behavior-preserving transformation. Do not rewrite the whole function. Extract one branch at a time.

Example: extract the new-user discount.

Before:

if (user.new) total = total * 0.9;

After:

total = applyNewUserDiscount(total, user);

function applyNewUserDiscount(total, user) {
  return user.new ? total * 0.9 : total;
}

Run the test suite again. All green. Then extract the shipping rule, then the promo rule.

Each extraction is a tiny commit. Each commit keeps the suite green. If something breaks, git bisect points to a two-line diff.

Step 5 — Run the Matrix on a Free Server (Optional)

Local node_modules can hide dependency issues. A disposable server gives you a clean environment. You can also run a larger test matrix without touching your machine.

I used MonkeyCode's free server option to run the same suite on a fresh system. I copied the project into the free server, ran node --test, and got the same green result. That confirms the tests are not coupled to my local setup.

This step is optional. It adds confidence when the legacy code depends on environment-specific globals.

Limitations

Characterization tests do not fix bugs. They freeze them in place. If the current output is wrong, your test will lock in the bug. You need an explicit decision to change behavior separately.

AI-generated edge cases are suggestions. They are not domain truth. A model cannot know your business rules. Always review each proposed input against the actual contract.

Who should not use this? Solo developers rewriting a small script probably do not need the ceremony. Teams shipping financial or safety-critical logic must pair characterization with human review. Never rely on a snapshot as your only safety net.

Conclusion

Characterization-first refactoring turns a messy function into a safe edit. Lock behavior with tests. Expand input coverage with a model. Execute the smallest change. Your repo becomes editable again.

Not sure where to start? Find the module with the most implicit rules. Write one characterization test for it today.

Diff the AI's Test Draft Before You Adopt It

Dakota Huang — Sat, 29 Aug 2026 21:24:42 +0000

Diff the AI's Test Draft Before You Adopt It

An AI-drafted characterization test is a hypothesis. Adoption without verification is a gamble. Draft, run, diff, adopt. That four-step loop is the whole job. Here is a working harness for it.

Why a Draft Looks Correct

A messy module has no tests. You want to refactor it safely. You ask a free model to draft characterization tests. The output looks professional. It reads like truth. Truth requires evidence.

Models interpolate patterns. They do not execute your code. Their training data contains thousands of tax functions. None of them is yours.

Draft tests fail in three predictable ways:

Invented expectations. The model asserts values no real call ever produced.
Bug normalization. The model "fixes" what it sees. The draft encodes intent, not behavior.
Context blindness. The draft uses inputs your code never receives.

All three produce a green test. Green is not evidence. Green only means the draft agrees with itself. Every failure mode survives pytest. The suite passes. The suite is wrong. The diff loop exists because passing is cheap.

Step 1: Record Fixtures from Current Behavior

Fixtures are recorded observations, not opinions. Sample real inputs. Capture real outputs. Store them as JSON Lines.

Choose inputs that touch each branch. Coupon present. Coupon absent. Empty cart. One item. Non-integer quantity, if the schema allows it. Five varied samples catch more deltas than fifty similar ones.

# record_fixtures.py
import json
import order

samples = [
    {"items": [{"price": 10.0, "qty": 2}], "coupon": None},
    {"items": [{"price": 4.99, "qty": 1}], "coupon": "SAVE10"},
    {"items": [], "coupon": None},
]

with open("fixtures.jsonl", "w") as fh:
    for sample in samples:
        row = {"input": sample, "output": order.total_with_tax(sample)}
        fh.write(json.dumps(row) + "\n")

Step 2: Draft Tests with a Free Model

Drafting is an iteration problem. Most candidates get discarded. Cheap iteration matters here. This drafting loop runs on MonkeyCode's free model access through the free server option. No local GPU needed. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The draft prompt needs a hard constraint. Verifiable tests require a predictable shape.

Write each test case as:
assert total_with_tax(<input literal>) == <expected literal>
No fixtures. No mocks. No loops.

That shape is what makes automated review possible. Free the format, and verification breaks.

Generate three draft candidates. Never settle for the first. Each candidate costs almost nothing. Run all three through the harness. Merge the passing cases into one file. That file is the candidate suite.

Step 3: Diff Draft Expectations Against Reality

The harness parses the draft's assertions. It compares each expected value to a recorded fixture. It prints one verdict per case.

# verify_draft.py
import ast
import json
import subprocess
import sys
from pathlib import Path


def draft_expectations(path):
    tree = ast.parse(Path(path).read_text())
    rows = []
    for node in ast.walk(tree):
        if not isinstance(node, ast.Assert):
            continue
        cmp = node.test
        if not isinstance(cmp, ast.Compare):
            continue
        left, right = cmp.left, cmp.comparators[0]
        if isinstance(left, ast.Call) and isinstance(right, ast.Constant):
            rows.append((left.args[0], right.value))
    return rows


def load_fixtures(path="fixtures.jsonl"):
    return [json.loads(line) for line in Path(path).read_text().splitlines()]


def main():
    observed = {
        json.dumps(f["input"], sort_keys=True): f["output"]
        for f in load_fixtures()
    }
    print(f"{'input':<46} {'draft':<14} {'observed':<14} verdict")
    for inp, expected in draft_expectations("test_order_draft.py"):
        key = json.dumps(inp, sort_keys=True)
        actual = observed.get(key)
        if actual == expected:
            verdict = "MATCH"
        elif key in observed:
            verdict = "DELTA"
        else:
            verdict = "NO_FIXTURE"
        print(f"{json.dumps(inp)[:46]:<46} {expected!s:<14} {str(actual):<14} {verdict}")

    run = subprocess.run(
        [sys.executable, "-m", "pytest", "test_order_draft.py", "-q"],
        capture_output=True,
        text=True,
    )
    print(f"pytest exit code: {run.returncode}")


if __name__ == "__main__":
    main()

Here is the output shape on a messy module:

input                                            draft      observed   verdict
{"items": [{"price": 10.0, "qty": 2}], "coupon": null}  21.6       21.6       MATCH
{"items": [{"price": 4.99, "qty": 1}], "coupon": "SAVE10"}  5.39  4.85       DELTA
{"items": [], "coupon": null}                             0.0        0.0       MATCH
pytest exit code: 1

The DELTA row is the model guessing intent instead of reading behavior. The model inferred that a coupon discounts the total. The real code discounts the subtotal first. Both are plausible. Only one is true.

Inspect every DELTA manually. Read the function. Trace the inputs. Ask one question: did the model misread the code, or does the code betray its own intent? The answer decides the fix. If the model is wrong, delete the case. If the code is wrong, write a spec test and annotate the bug.

The parser only reads literal comparisons. Other shapes are skipped silently. Keep the prompt constraint strict, or extend the parser.

Step 4: Apply the Mutation Gate

A passing draft test can assert nothing useful. Prove usefulness the cheap way. Break behavior on purpose. The suite must fail.

# Change the tax rate. Nothing else.
sed -i 's/0.08/0.09/' order.py
python -m pytest test_order_draft.py -q   # must show failures
git checkout order.py

One intentional break. Multiple failures. That is the mutation gate. Run one break per gate pass. Two breaks can hide each other.

Reading the Verdicts

Verdict	Meaning	Action
MATCH	Draft equals a real observed output.	Keep the test.
DELTA	Draft disagrees with reality.	Inspect. The draft may be wrong. The code may be buggy.
NO_FIXTURE	No recorded input reached this path.	Record a fixture or drop the test.

Expect DELTA verdicts in a messy module. They are not noise. They are the review. A draft with zero deltas is either excellent or untested. Assume untested. A DELTA usually fails pytest. That failure is the review working.

Adoption rules before you merge:

Every adopted test has a MATCH verdict.
Every MATCH has a fixture.
The mutation gate failed once per intentional break.
No test entered the suite without a verdict.

Mechanical rules beat taste. Encode them in CI if you can. The harness is a script. Scripts do not get tired.

Then Make the Smallest Safe Change

Verified tests are your behavior contract. Make one move. Commit the suite first.

# one move: extract the discount decision
def discount_rate(order):
    return 0.9 if order.get("coupon") == "SAVE10" else 1.0


def total_with_tax(order):
    subtotal = sum(i["price"] * i["qty"] for i in order["items"])
    return round(subtotal * 1.08 * discount_rate(order), 2)

Run the suite. Zero failures means behavior held. Commit the suite. Commit the move. Do not merge the two commits.

Limitations and Who Should Skip This

This loop verifies current behavior. It never verifies correct behavior. If the code is the bug, characterization tests protect the bug. Write spec tests instead.

Skip this workflow when:

The code is greenfield. Write the spec first.
The refactor must change behavior. Characterization locks the old contract.
The draft will be merged unreviewed. A draft is a hypothesis.

Fixtures decay. Inputs change. Recapture them when the suite goes green too easily. A stale fixture is a false memory.

The free-server side is an availability claim, not a guarantee. Model identity, quotas, and uptime change. None are verified here. The harness works with any drafting source. Verification stays local and controllable. That split is the point.

The Score That Matters

Run the loop on your messiest module. Count the DELTA verdicts. The count is your review quality score. Post it. Compare notes with other reviewers.

Adoption sequence:

Record fixtures.
Draft three candidates.
Diff every expectation.
Kill the DELTA rows.
Run the mutation gate.
Commit the suite.
Make one move.
Run again. Commit.

The AI Wrote the Diff. The Tests Wrote the Verdict.

Dakota Huang — Sat, 29 Aug 2026 18:20:55 +0000

The AI Wrote the Diff. The Tests Wrote the Verdict.

AI refactor suggestions are hypotheses. Not facts. A free coding model rewrites your messy legacy function. The diff looks clean. CI stays green. Then a customer hits an edge case you forgot.

This article shows a small workflow. Characterize legacy behavior first. Let the model propose a refactor. Run the same tests against both versions. The verdict: safe or not safe.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why Characterization Comes First

Legacy code has no spec. The only reliable spec is current behavior. Even bugs are behavior. If your refactor changes a bug, you need to know.

A characterization test records inputs and outputs. It does not judge right or wrong. It freezes the current contract. After freezing, every difference becomes visible.

Step 1: Capture Real Inputs and Outputs

Pick one messy function. I used a shipping calculator. Nested conditionals, magic numbers, zero tests.

Write a probe script. Call the function with realistic cases. Save outputs as JSON.

import json
from legacy import calculate_shipping

cases = [
    {'items': [{'weight': 2.0, 'qty': 3}], 'region': 'US'},
    {'items': [{'weight': 0.5, 'qty': 10}], 'region': 'EU'},
    {'items': [{'weight': 0.2, 'qty': 1}], 'region': 'US'},
    {'items': [{'weight': 5.0, 'qty': 2}], 'region': 'JP'},
]

for c in cases:
    result = calculate_shipping(c['items'], c['region'])
    print(json.dumps({'input': c, 'output': result}))

Save output to captured.json. That becomes ground truth.

Step 2: Ask the Model for a Refactor

MonkeyCode's free model access lets me prompt from the CLI. I gave the model one strict instruction: keep behavior identical.

Refactor calculate_shipping into smaller functions.
Do NOT change edge cases. Do NOT change rounding.
Extract private helpers only.

The model returned a diff. It split the function into three helpers. The logic looked clean. Clean looks are not evidence.

Save the refactored version as shipping_refactored.py. Leave the original untouched.

Step 3: Turn Captures Into Tests

Load captured.json. Write a characterization test that runs against either version. The assertion is simple: same input, same output.

import json
import pytest
from legacy import calculate_shipping
from shipping_refactored import calculate_shipping as refactored

with open('captured.json') as f:
    cases = json.load(f)

@pytest.mark.parametrize('case', cases, ids=range(len(cases)))
def test_characterization(case):
    expected = case['output']
    assert calculate_shipping(case['input']['items'], case['input']['region']) == expected

Run the test on the original. It passes. Of course it does. It measured the original.

Point the same test at the refactored version. Now the diff matters.

Step 4: Run a Differential Test

Parametrized tests only compare to captured values. A differential test compares old and new directly. It finds divergence.

@pytest.mark.parametrize('case', cases)
def test_differential(case):
    args = case['input']
    old = calculate_shipping(args['items'], args['region'])
    new = refactored(args['items'], args['region'])
    assert old == new, f'mismatch for {case}'

My first run showed one mismatch. A single item under 0.3 kg in the US region. The model had changed weight <= 0.5 to weight < 0.5. One byte flipped a shipping charge from $0 to $4.99.

The test caught it. The model's diff looked perfect. The evidence said otherwise.

Step 5: Push Harder With Property-Based Inputs

Recorded cases cover what you saw. They do not cover what you missed. Property-based testing generates new inputs and searches for contradictions.

from hypothesis import given, strategies as st

Item = st.fixed_dictionaries({
    'weight': st.floats(min_value=0.01, max_value=50.0),
    'qty': st.integers(min_value=1, max_value=20),
})

@given(items=st.lists(Item, min_size=1, max_size=5),
       region=st.sampled_from(['US', 'EU', 'JP']))
def test_property_differential(items, region):
    old = calculate_shipping(items, region)
    new = refactored(items, region)
    assert old == new

I ran 1000 generated cases. The differential test found 17 failures. All sat near the same weight boundary. The model introduced a systematic off-by-one error.

Property tests do not prove equivalence. They prove divergence under search. That is enough to reject a refactor.

Performance and Server Note

I used MonkeyCode's free server option to run the harness. It kept the experiment isolated from my local environment. No CI machine needed for a 20-test suite.

Do not treat the free server as a production runner. It is scratch space. Free tiers change. Quotas are not guarantees.

Limitations

Characterization tests freeze behavior on collected samples. Missing samples mean missing boundaries. Add property-based guessing. Review the diff by hand.

Tests ignore performance. Identical values can still be 10x slower. Add a benchmark if latency matters.

Exception semantics matter. Two functions can return the same values but raise different exceptions. My tests did not cover exception paths. Yours should.

Who Should Skip This Workflow

Skip one-off scripts. Characterization adds ceremony you do not need.

Skip fresh code with a real spec. Write behavior tests from the spec.

Skip it if you cannot review the model's diff. The workflow only finds differences. It does not judge which behavior is correct. You still have to think.

Bottom Line

The model made refactoring cheap. Tests made it safe. AI reviews your code. Nobody tested the reviewer. Characterization tests are how you do it.

Capture behavior. Write tests. Run both versions. Let the diff prove itself.

The free model gives you speed. The free server gives you isolation. The tests give you the truth. Use all three.

A Refactor Without a Baseline Is a Trust Fall

Dakota Huang — Sat, 29 Aug 2026 15:16:14 +0000

A refactor without a baseline is a trust fall. You close your eyes and hope. The hope usually lands somewhere else.

AI reviewers are good at reading diffs. They cannot read your intent. A diff review checks shape. It misses behavior changes. You need an external memory. Characterization tests are that memory.

The baseline contract

Fix the contract before you refactor. Pick one messy pure function. Keep it small. Avoid time, random, network, and file I/O. Pure functions give stable baselines.

The contract has three parts. Inputs go in. Outputs come out. Exceptions are also outputs. Snapshot all three.

Step 1: Write the snapshot runner

Put this script outside the function file. It loads a module, runs named cases, and writes JSON.

# snapshot.py
import json
import sys


def run(path, cases):
    ns = {}
    with open(path) as f:
        exec(compile(f.read(), path, "exec"), ns)

    results = []
    for c in cases:
        item = {"name": c["name"]}
        try:
            item["ok"] = True
            item["result"] = repr(
                ns[c["fn"]](*c.get("args", []), **c.get("kwargs", {}))
            )
        except Exception as exc:
            item["ok"] = False
            item["exception"] = type(exc).__name__
            item["detail"] = str(exc)
        results.append(item)
    return results


if __name__ == "__main__":
    cases = json.load(open(sys.argv[2]))
    result = run(sys.argv[1], cases)
    json.dump(result, open(sys.argv[3], "w"), indent=2)

The runner never imports your production package. It compiles one file. That keeps the harness safe for old code.

Step 2: Design probe cases

Name every case. Unnamed cases make diff reading painful. Cover the edges that old code tends to hide.

[
  {"name": "two items", "fn": "calculate_total", "args": [2, 10.0]},
  {"name": "zero quantity", "fn": "calculate_total", "args": [0, 10.0]},
  {"name": "negative price", "fn": "calculate_total", "args": [1, -2.0]},
  {"name": "missing price", "fn": "calculate_total", "args": [1, null]}
]

Start with five to ten cases. You can expand later. More cases means more signal. Most of the value comes from edge cases.

Step 3: Record the baseline

python3 snapshot.py order.py order_cases.json baseline.json

Commit baseline.json first. The commit message can say exactly what you are preserving. Reviewers get a concrete artifact. Nobody has to trust the old function's reputation.

Step 4: Generate the refactor against the contract

Now ask a model to rewrite the function. Give it the baseline file. Give it the case file. Tell it the before/after output must match exactly.

The model can propose a new version. The baseline is not a suggestion. It is a pass/fail gate.

Step 5: Run the new code and diff

python3 snapshot.py order_new.py order_cases.json after.json
diff baseline.json after.json

A clean diff means behavior survived. A noisy diff shows exactly which case broke. This turns a vague review note into a reproducible test.

Which functions deserve a snapshot?

Function trait	Snapshot before AI edits?	Why
Fan-in greater than 3	Yes	Many callers broaden the blast radius.
Many branches or early returns	Yes	Bugs hide in branch paths.
Time, random, or network calls	No	The baseline will flap.
Trivial accessor	No	The snapshot adds noise.

Use the table before every refactor. A cheap snapshot on the wrong function costs maintenance time. No snapshot on the right function costs an incident.

Why the free options matter

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The generate step in this loop needs an AI backend. I used MonkeyCode's free model access for that step. It fits because the refactor iteration is the part that can cost money. A free tier removes the anxiety from retrying. The free server option matters for the other side. Some repos cannot leave a private network. A local server keeps the baseline, the candidate, and the diff in the right place.

The value of the workflow does not depend on MonkeyCode. The script, the cases, and the diff are plain JSON. That is the point.

Limitations

Snapshots preserve bugs. If the original output is wrong, the baseline is wrong. They catch behavior changes, not design problems. They are not a replacement for real tests. They are a wrapper around old behavior.

Use this only on pure functions first. For stateful code, snapshot the input object too. For I/O-heavy functions, build explicit fixtures. Otherwise the baseline lies.

Who should not use this

Teams with full branch coverage should skip the snapshot. A well-tested function already has a baseline. Snapshotting it adds a second version of the same test.

People who want a fully automated refactor should also skip this. The model still makes choices. You still read the diff. The loop only removes the guesswork.

And never write PII into a snapshot file. Characterizing a legacy function does not justify leaking user data.

Next time you ask a model to clean up a mess, record the behavior first. Then the review is a diff. The refactor stops being a trust fall.