The Impact Radius: Refactor Around Every Caller You Can't See

Your unit tests pass. Your staging smoke passes. Then a plugin crashes at 3 a.m. because it called the function the way the docs never mention. The function was simple to refactor. The callers were not.

This is not a test failure. It's a blind spot. You changed a function, but the real contract lives in every call site that already knows how to use it. Tools will not find that contract for you. So you need a snapshot discipline.

Call this the impact radius: the set of places that reach into the function, pass weird shapes, rely on an undocumented default, or treat an exception as normal. The only safe refactor is the one that preserves the impact radius, not just the unit tests.

Here is a complete workflow you can run today. It uses a small AST scanner, a free AI model to infer argument shapes, and a disposable server to run a before/after snapshot. No production database, no Docker daemon, no waiting for CI.

Step one: find the impact radius. Start with a function that has suspicious flexibility. Take this inventory merger:

def merge_stock(current, incoming):
    if isinstance(incoming, list):
        for item in incoming:
            current[item['sku']] = current.get(item['sku'], 0) + item['qty']
    else:
        for sku, qty in incoming.items():
            current[sku] = current.get(sku, 0) + qty
    return current

It accepts two shapes for incoming: a dict of sku→qty, or a list of items. That smells. But the fix is not obvious until you know who sends which shape. Run a search.

rg -n 'merge_stock' .

Then automate the search with an AST script. Grep shows you lines, not shapes. You need the actual expressions. This script walks every Python file and prints call sites and argument AST nodes.

import ast, pathlib, sys

target = sys.argv[1]
for path in pathlib.Path('.').rglob('*.py'):
    tree = ast.parse(path.read_text())
    for node in ast.walk(tree):
        if isinstance(node, ast.Call):
            fn = node.func
            if isinstance(fn, ast.Name) and fn.id == target:
                print(f'{path}:{node.lineno}')
                for arg in node.args:
                    print('  arg:', ast.dump(arg)[:160])

For a real repo, the output looks like this:

store/core.py:41
  arg: Name(id='inventory')
  arg: Call(func=Attribute(attr='values'), ...)
plugins/restock.py:83
  arg: Attribute(attr='stock')
  arg: List(elts=[...])

Now you have the raw ingredients. The next step is the interesting one: turn those AST fragments into a parameter contract. This is where MonkeyCode's free models are genuinely useful. Shape inference is a small, contained prompt—not a place where you need an expensive frontier model.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A prompt like this works:

Here are call sites for merge_stock. For each, infer the runtime type of every argument (dict, list of objects, tuple, None). Return JSON only, with the line number as key.

The model reads the surrounding code and produces a shape map. You scan it, correct the guesses, and keep it as the contract. The shape map is a tiny piece of documentation. You can paste it above the function or commit it as shape_map.json. The point is not elegance. The point is that the next refactor starts from evidence, not anecdotes.

Then you create a call-site snapshot. This is a snapshot written from the perspective of each inferred caller, not from the function's own happy path.

import json
import inventory

snapshots = {
    'core_inventory_values': inventory.merge_stock({'a': 1}, {'b': 2}),
    'plugin_restock_list': inventory.merge_stock(
        {'a': 1}, [{'sku': 'a', 'qty': 3}]
    ),
    'legacy_tuple_abuse': inventory.merge_stock(('a', 1), {'b': 2}),
}
print(json.dumps(snapshots, sort_keys=True))

Run it before the refactor and save the output.

python call_site_snapshot.py > before.json

This is the point where the free server option helps. Instead of setting up a fresh venv on your laptop, you can use MonkeyCode's free server option to run the snapshot in a disposable environment. The workflow stays reproducible: checkout the commit, install deps, run the script. Nothing leaks into your local Python state.

Now refactor the function with a minimal structural change. For example, extract the two branches into helpers. Do not change the public signature. Do not 'improve' edge cases. The goal is to preserve behavior exactly, so the snapshot becomes the lock.

def _merge_items(current, items):
    for item in items:
        current[item['sku']] = current.get(item['sku'], 0) + item['qty']

def _merge_mapping(current, mapping):
    for sku, qty in mapping.items():
        current[sku] = current.get(sku, 0) + qty

def merge_stock(current, incoming):
    if isinstance(incoming, list):
        _merge_items(current, incoming)
    else:
        _merge_mapping(current, incoming)
    return current

Notice what stayed the same: the branching on isinstance, the mutation of current, the return value. Even the order of mutation remains identical. That is the point of a structural refactor.

Now run the snapshot again.

python call_site_snapshot.py > after.json
diff -u before.json after.json

If diff is empty, you have evidence, not vibes. The refactor did not change the output for every shape the invisible callers used. If diff is noisy, you found the exact contract you were about to break.

This is a small workflow, but it changes where trust lives. You no longer trust the AI to understand the whole system. You trust a snapshot that was built from the real call sites and confirmed on a clean server. The AI does the shape inference, the robot does the bookkeeping.

Limitations are honest about this method. The AST scanner finds static calls only. Anything reached through getattr, eval, dynamic imports, or a framework's plugin loader will stay invisible. The AI shape inference is probabilistic; if you skip the manual scan of the shape map, you turn it into a hype generator. And the snapshot itself is not property-based testing. It proves those inputs, not all inputs.

If your snapshot exposes a weird tuple being passed as current, that is not a refactoring problem. That is a contract problem. Fix the contract first, then refactor. Skip this workflow when call sites are dynamically generated, when the code mutates external state (files, queues, sockets), or when you have no command to run the snapshot reproducibly. In those cases, a hand-written unit test gives false confidence.

But for a typical Python codebase with visible calls, the impact radius is small enough to snapshot in an afternoon. Start with one function and one suspicious branch. Find the callers you cannot see. Lock their behavior before any model touches the code.

Run the AST scanner on your own repo today. You may find a caller you forgot existed. Next time a refactor feels simple, ask yourself: how many callers am I not looking at? If the answer is 'I don't know,' the AI can draft the patch, but the snapshot is the only thing that can approve it.