DEV Community

Dakota Huang
Dakota Huang

Posted on

You're the Reviewer Now. Who Reviews Your Verdict?

AI turned every developer into a reviewer. Nobody validates the reviewer. When an AI suggests a refactor, your approval is the gate. Without a behavior baseline, your verdict is a guess.

This post explains how to test your own review. It uses characterization tests to lock current behavior. Then it measures an AI diff against that lock. The method is repo-agnostic and costs almost nothing.

The reviewer's blind spot

You see a diff. The AI calls it a refactor. The test suite is green. You approve. Weeks later, a regression appears.

The tests were written for the old shape. They rarely cover the edge cases a refactor touches. Your review needs a second signal.

Characterization tests provide that signal. They record what the code does today, not what it should do. They turn implicit behavior into executable assertions.

Step 1: Record the behavior fingerprint

Take a legacy function. Here is a typical price parser:

# legacy.py
def parse_price(value):
    if value is None:
        return 0
    if isinstance(value, str):
        return float(value.replace("$", "").replace(",", ""))
    return float(value)
Enter fullscreen mode Exit fullscreen mode

Write a characterization test for the input shapes you know exist:

import unittest
from legacy import parse_price

class TestParsePrice(unittest.TestCase):
    def test_existing_behaviors(self):
        cases = [
            ("$1,234.50", 1234.5),
            (None, 0),
            (" 42 ", 42.0),
            ("0", 0.0),
            ("-7", -7.0),
        ]
        for raw, expected in cases:
            with self.subTest(raw=raw):
                self.assertEqual(parse_price(raw), expected)
Enter fullscreen mode Exit fullscreen mode

Run it on the legacy code. It must pass. If it fails, your test does not match production behavior. Fix the test, not the code.

Step 2: Draft edge cases with a free model

Five cases are not enough. You need more input families. A free model can propose candidates quickly.

For example, MonkeyCode's free model access can draft a table of edge cases in seconds.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat that draft as raw material, not truth. Verify each candidate against the legacy behavior. This is the same rule you apply to the refactor. AI output never skips the verification step.

Step 3: Diff the implementations

Now you receive a proposed refactor. A common simplification is:

# new_impl.py
def parse_price(value):
    return 0 if value is None else float(str(value).replace("$", "").replace(",", ""))
Enter fullscreen mode Exit fullscreen mode

Run your characterization tests against both versions. Then run a differential check. Use a generator that draws from real call-site patterns, not pure randomness:

import random

def realistic_input():
    r = random.random()
    if r < 0.1:
        return None
    if r < 0.6:
        return f"${random.randint(0, 100000):,}"
    return random.uniform(-1000, 1000)

for _ in range(2000):
    raw = realistic_input()
    old = parse_price_old(raw)
    new = parse_price_new(raw)
    if old != new:
        print("divergence:", repr(raw), old, new)
        break
Enter fullscreen mode Exit fullscreen mode

If you want to run this loop without touching your production environment, MonkeyCode's free server option is a low-friction place to do it. Spin up a scratch environment, paste both functions, run the generator. The result is a pass or a divergence list.

Step 4: Make a verdict

Use a decision table. This is the core of reviewing the reviewer.

Signal Verdict
Characterization tests pass on both versions Proceed to manual diff review
Differential test finds a divergence Reject; ask AI to preserve old behavior
Diff touches a function with fan-in > 5 Reject unless you add caller tests
No characterization baseline exists Reject before reading the diff

The last row is the most common failure. Teams approve AI refactors because tests are green. The tests were too weak to notice the change.

Limitations

This workflow protects only observable behavior. It misses concurrency, timing, and external side effects. It also struggles when the current behavior is a bug you want to fix. In that case, write a regression test for the new behavior first.

Who should skip this? Solo developers on throwaway prototypes. If the code dies in a week, characterization overhead is waste. Use it when the code has real callers.

The honest verdict

Your review is as strong as the behavior lock beneath it. Free model access and a free server make the loop cheap, but they do not replace judgment.

Next time you accept an AI refactor, ask one question: "What did I lock before I approved?" If the answer is nothing, you did not review. You guessed.

Top comments (0)