There’s a specific kind of dread that comes with opening a file called pricing_v2_final_FIXED.py and realizing you’ve been asked to change it. No tests. Comments that contradict the code. A function that’s 400 lines long and returns slightly different things depending on a flag nobody remembers adding. You could try to understand all of it before touching anything. That might take a month.
Golden master testing offers a different deal. You don’t have to understand the code to protect it. You just have to record what it does right now, and then make sure it keeps doing that while you clean it up. For APIs in particular, it’s one of the fastest ways to go from “terrified to touch it” to “refactoring with a safety net.”
What golden master testing actually means
The name comes from the audio world, where a golden master was the final approved recording every copy got checked against. In software it works much the same way. You run the system with a set of inputs, capture everything it produces, and save that output as the approved version. From then on, every change gets run against the same inputs, and any difference from the saved output fails the test.
You’ll also hear it called characterization testing, a term Michael Feathers popularized in Working Effectively with Legacy Code, or approval testing, or snapshot testing. Details differ a bit between them. The core idea doesn’t: the current behavior is the spec.
That last part trips people up. A golden master doesn’t claim the current behavior is correct. If the legacy pricing API rounds totals in some strange way, the golden master captures the strange rounding. That’s the point. Before a refactor, you want to know about every change, intended or not, and decide later which ones are bugs worth fixing.
Why it fits APIs so well
APIs are almost tailor-made for this. Inputs are requests, outputs are responses, and both are easy to capture and compare. You don’t need to reach inside the code at all, which is exactly what you want when the code is the scary part.
There’s also the matter of clients. Somewhere out there, a mobile app or a partner integration depends on how your legacy API behaves today, quirks included. A golden master protects those clients from changes nobody meant to make, which is usually more valuable than protecting some idealized version of the API that exists only in documentation.
Choosing inputs that actually exercise the code
A golden master is only as good as its inputs. Feed it three hand-picked requests and you’ve protected three paths through a system that probably has hundreds.
Real traffic is the best source by far. Production logs, API gateway records or captured staging traffic show exactly what clients send, including the odd parameter combinations and half-empty payloads nobody would think to type out. Pull a broad sample, scrub anything personal, and you’ve got inputs that reflect how the API is really used.
If you want a way to check whether the inputs are good enough, measure code coverage while you run them. Branches that never execute are paths the golden master isn’t protecting. Sometimes that’s fine, like admin features nobody uses. Sometimes it reveals a whole pricing rule your sample missed, and it’s time to add a few deliberate requests to cover it.
Capturing more than the response
Here’s where API golden masters get interesting. The response body is the obvious thing to capture, but plenty of legacy behavior never shows up in a response.
A request might write rows to a database, publish a message to a queue or call some third-party API. If the refactor changes any of that, a response-only golden master won’t notice. So it’s worth capturing side effects too: what got written, what got sent, what got called. Comparing database state before and after each request catches a whole category of regressions that would otherwise slip right through.
Capturing outbound calls has a second benefit. Once you’ve recorded what the database and external services returned, you can replay those answers during testing instead of hitting live systems. The golden master then runs the same way every time, without depending on a shared staging database that someone else might reset.
Dealing with output that changes every run
Run almost any API twice and the responses won’t match exactly. Timestamps differ. IDs get regenerated. Session tokens rotate. Compare those byte for byte and the golden master fails on every single run, which makes it useless.
The fix is scrubbing: replacing known volatile values with stable placeholders before comparing. Swap every timestamp for something like a fixed marker, every generated ID for a numbered placeholder, and comparisons start working again. The trick is scrubbing only what genuinely varies. Scrub too much, like ignoring a whole object because one field inside changes, and real regressions sneak through unnoticed.
Refactoring under the golden master
With the golden master in place, the actual refactor gets much calmer. Make one small change. Run the golden master. If nothing differs, keep going. If something does, you know exactly which change caused it, because you only made one.
When a difference shows up, decide on purpose. If it’s a bug the refactor introduced, fix the code. If it’s an intentional improvement, approve the new output so it becomes the baseline. Approving should be a deliberate step that someone reviews, ideally in the same pull request as the code change, never a reflex to make the build go green.
Sometimes the golden master surfaces an old bug, like a total that’s been off by a cent for years. Resist fixing it mid-refactor. Note it, keep the old behavior pinned for now, and fix it in its own change, where the behavior change is the whole point and clients can be warned.
When to retire the golden master
Golden masters are scaffolding, not the final building. Once the code is understood and cleaned up, focused tests that check specific behavior tend to be more readable and easier to maintain than a giant pile of recorded outputs.
That doesn’t mean throwing the golden master away the moment refactoring ends. Plenty of teams keep a slimmed-down version running as an API regression suite, since it’s still great at catching anything a client would notice. What changes is that it stops being the only safety net.
Tools that help
A few tools support the golden master approach in different ways:
- ApprovalTests is a library available for many languages. It saves approved outputs as files, shows diffs when they change, and includes scrubbers for volatile values.
- Keploy works at the API level. It records real API traffic along with the database and downstream calls behind each request, then replays everything as tests with those dependencies mocked, which makes it an automatic golden master for APIs with no test scripts to write.
- Jest snapshot testing brings the same idea to JavaScript and TypeScript projects, storing snapshots next to tests.
Conclusion
Legacy code doesn’t stay scary because it’s complicated. It stays scary because nobody can tell whether a change broke something. Golden master testing fixes that without asking anyone to understand the code first. Capture what the API does today, scrub what naturally varies, and every refactor becomes something you can check instead of something you hope about.
That 400-line pricing function might still be ugly a week from now. The difference is that someone will finally be able to change it, one small step at a time, knowing exactly what each step did.
Top comments (0)