DEV Community: Anton Brilliantov

Breaking the Monolith - Part 1: A 200 OK That Saved Nothing

Anton Brilliantov — Mon, 10 Aug 2026 10:09:37 +0000

How a single spurious was_updated exposed a hidden rule about ownership - when a system is split, exactly one place is allowed to own identity. This is the expanded cut of Part 1: the debugging and the fix, plus the piece the first version only gestured at - the staged migration this bug lives inside, and where the seam goes right after it.

👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go. For a while now I've been on the kind of project that teaches you the most: carefully breaking a large PHP monolith into Go microservices while it's still very much alive and serving a real business. This is an expanded re-cut of Part 1 - same bug, same fix, but with the migration map drawn in properly, because half the lesson is which stage you're standing in when the ground moves. The running notes live on my GitHub: github.com/brilliant-almazov. No hype, just the real work.

A teammate pinged me with a vague, ordinary-sounding report: domain markup for a region was coming back empty in a snapshot. Not an error, not a stack trace - just wrong output a human noticed because the business process it feeds looked off. I reproduced the write against production and the API told me, cheerfully, that everything was fine - 200 OK, action: was_updated, a response body carrying the correct, freshly computed rule set. And yet nothing had changed in the database. updated_at frozen. The downstream materialized count flat. A silent no-op wearing a success badge.

That one bug turned out to be the perfect lens on the whole migration I'm living in - but you can't see why until you can see the stages. So before the code, let me draw the map.

The migration, in stages

A big-bang rewrite is a bet you make once and lose slowly. The alternative - the one that actually works on a live business - is a strangler fig: move the seam one named stage at a time, and never let two migrations run at once. Here is the seam this whole series lives on - the rule-set write path - laid out as stages, with a marker on where this bug happened:

Stage 0  Monolith only.
         PHP computes the cascade, persists the rule data, owns identity implicitly.

Stage 1  Identity extracted.          <-- THIS ARTICLE
         A small Go service becomes the immutable, content-addressed master of
         identity (id + hash). PHP still computes and persists the rule DATA.
         The bug: the monolith and the new identity master quietly DISAGREE.

Stage 2  Write logic extracted.       (Part 1.5 — next)
         The cascade is re-implemented in a stateless Go service that writes
         directly into the monolith's DB. Two engines run in parallel.

Stage 3  Prove they agree.
         An agent drives both engines and diffs persisted truth, case by case.

Stage 4  Flip, then delete.
         Promote the Go engine, retire PHP, and only then move the DATA out.

This article is Stage 1 - the first moment a second system has an opinion about identity. Everything downstream (the parallel run, the cutover) is only safe if this stage is honest. It wasn't yet.

Context: what this is and how it works

Strip away the specifics and the domain is simple. The system stores classification rules - patterns that tag web domains with a meaning ("this domain is ours," "this one is a competitor," "this one is irrelevant"). A rule is roughly {pattern, type, match-mode, priority}. A rule set is an ordered collection of those rules attached to one node in a hierarchy.

The hierarchy has three levels, and it cascades top-down:

CLIENT            rules here apply to everything beneath
  └── PROJECT     rules here apply to every config of the project
        └── CONFIG   a specific target (search-engine × device × locale)

Writing a rule set at a parent level doesn't just save that node - it fans out: the engine recomputes every descendant's effective rule set by merging what it inherits from above with what it owns locally, with a clear precedence (a narrower level overrides a broader one). The per-node result of that merge is the materialized state: the fully-resolved set of rules a given config actually sees at request time. That snapshot is what the business runs on.

Two concepts matter for the bug:

origin - a label on each rule in a materialized set saying which level it came from (client / project / own). Descriptive metadata about the cascade.
identity - the answer to "is this the same rule set or a different one?" Historically an internal detail of the monolith. It stops being an internal detail the moment a second system needs to agree on it. Hold that thought - it's the whole article.

Now the Stage-1 overlay, and this is the part that took me a couple of iterations and a correction from a colleague to state correctly. The write logic - the cascade, the validation, the transactional integrity - lives in the PHP monolith. A newer Go service, the domain-rule-map service (I'll also call it the rule-set store when its identity role is the point), has been introduced as the authority for rule-set identity. It is deliberately, aggressively dumb: immutable and content-addressed. You hand it a rule set, it hashes the content, and it either returns the existing master for that hash or creates a fresh one. It creates exactly what you give it - nothing more. The data and the cascade logic stay in the monolith; the store owns only the id/hash, in its own database.

There's a subtlety I initially got wrong, and a teammate set me straight in one sentence: creating a rule set is not the same operation as recalculating. Creation is send the computed set to the store + save our mirror, transactionally, in one shot. Recalculation is a separate, post-commit process that fans out over everything that changed. Conflating the two - treating the store's echo of a create as the source of truth for our content - is a category error, and it's exactly the kind of thing that reads fine in a diagram and detonates in production.

So at Stage 1 the monolith computes and writes, the store owns identity, and both back the same reads. Every actor now has to agree on two things: the content of a rule set, and whether a given write actually changed the persisted state. This bug is what happens when they quietly disagree about the second one.

How I got it wrong first - the iteration that matters

I want to be honest about the shape of the investigation, because the wrong turns are the lesson.

Iteration 0 - "there's no bug." My first conclusion, embarrassingly, was that nothing was broken. I had a green functional test exercising the write, and a production dry_run that returned the correct cascade. Both said fine. So I told the team it was probably stale data from deploy lag - re-submit and it'll re-materialize.

That was wrong for a precise, instructive reason. The test ran the legacy local write path. The dry_run only computes the cascade - it persists nothing. But production runs behind a runtime master-switch flag that routes writes through the store path instead. A different branch of code entirely. Neither of my two "proofs" had touched the path that was actually failing in prod. A green test on the wrong code path is worse than no test - it manufactures false confidence.

Iteration 1 - reproduce on the real path. The only way forward was an authorized, real (non-dry_run) write against a production polygon, on the store path, watching persisted state rather than the response. And there it was, undeniable:

RESPONSE  200 OK   action: was_updated
          config:15  ->  master A   (16 rules)     "looks great"

DATABASE  config:15  ->  master B   (8 rules)      the scope row points at B
          master A    ->  backs NOTHING            A was never linked

          the 200 described a world that does not exist in the DB

The monolith's computed identity and the store's persisted identity had diverged - and the scope row that ties a config to its rule set pointed at neither of the things the response bragged about.

Iteration 2 - the wrong hypothesis, corrected by a human. With the divergence in hand I built a tidy theory: the store must be origin-blind - it's collapsing two different sets into one master. I even took it to the team as an architecture question, floating the idea of changing the store. A colleague shut it down in one line: the store is immutable; it builds exactly what you pass it; if a rule didn't survive, the monolith is what dropped it. He was right, and it reframed the entire hunt. The fault had to be entirely on the monolith's side. That correction saved me from "fixing" the one component that was behaving correctly.

That's the part I most want to land: the tests were green, the AI-assisted code generation was fast, the diagram was clean - and the thing that turned the investigation was a person who knew the invariant and a person on the business side who noticed the output was wrong. Tooling found none of that. Judgment did.

The root cause, at the level of the code

Under the store path, three pieces of monolith code conspired to produce the silent no-op. None is exotic; all survive review because each looks locally reasonable.

1. The creator was insert-only. The store master id is stable per (scope, entity). When a rule got re-stamped - say origin: own → origin: project, identical pattern/type/match/priority - the content changed but the resolved id stayed the same. The creator looked the id up, found an existing master, and returned the stale local mirror as-is. Create had quietly become "create-if-absent, ignore otherwise."

2. Correlation was by content hash. After computing the cascade, a sync builder had to match each computed item to the persisted master it produced. It did that by re-hashing the computed content and looking it up:

// The fragile correlation: re-hash the (possibly mutated) content and hope it matches.
$persisted = $createdByHash[$newRuleSet->getHash()] ?? null;
if ($persisted === null) {
    continue; // ← silent drop. The scope row never re-links. Data lost, no error.
}

The monolith's hash included origin; the store's, at that point, did not. So the instant a re-stamp changed the local hash, the lookup missed, the item fell into a silent continue, and the scope row was never re-linked. Cascade descendants and shrinking removes vanished exactly this way - no exception, no log, no trace.

3. The action verdict was decided by a local hash, never by what persisted.

// The spurious verdict — decided entirely by the client's local, origin-inclusive hash.
$action = $old->hash() !== $new->hash()
    ? BatchUpsertAction::WasUpdated    // origin changed → local hash changed → "updated!"
    : BatchUpsertAction::WasUnchanged;

old.hash !== new.hash was true - origin had changed - so the endpoint stamped was_updated, while under it nothing had landed. Two hash functions is one hash function too many. Asymmetric identity between a client and its master is a silent-corruption generator: no status code will ever reveal it, because the code that writes the status never asks the database what happened.

And note again where the fault is not. The store did its one job correctly. This is the whole reason it was worth extracting: a small, immutable, content-addressed authority is easy to reason about. The mess was in the seam - the monolith's assumptions about a partner that had only just come into existence.

The fix, and the constraint it enforces

The fix had to do three things, and - the part I'm proud of - it had to do them without mutating a single existing interface or DI contract, because those contracts are shared with the paths that were already working. In Symfony, that's what the container is for.

Pattern 1 - one resolver, one method

A resolver resolves. It has exactly one method - resolve - and the fact that now matters (the store-assigned master that actually landed) enters as a nullable argument, not as a second method or a parallel executed/executedFor interface zoo:

declare(strict_types=1);

// One method. The persisted master is just an argument — nullable, because
// "nothing landed" is a real, first-class outcome that must be expressible.
interface ActionResolverInterface
{
    public function resolve(
        StateItemInterface $item,
        ?RuleSetInterface $persisted,
    ): BatchUpsertAction;
}

Every implementation is-an ActionResolverInterface everywhere it matters, so the runner routes by polymorphism, never by an instanceof ladder in shared code. Adding an implementation never touches the orchestrator.

Pattern 2 - carry the truth on the aggregate, captured at creation

The verdict needs one fact that didn't exist before: for a given computed item, what master did the store actually assign? The wrong way to deliver it is to thread a handle through five signatures. The right way is to capture it at creation time, keyed by the item's original input hash, and hang it on the aggregate the pipeline already passes around - behind a readable micro-interface:

// The deterministic calculated → persisted link, captured by submission
// alignment at creation — never re-derived from (divergent) persisted content.
interface PersistedMasterReadableInterface
{
    /** @return array<string, RuleSetInterface> keyed by input hash */
    public function createdByInputHash(): array;
}

The key move is when and by what key the link is captured: at the moment of creation, under the original input hash, before any re-stamping can make the content diverge. That single decision makes correlation deterministic instead of hopeful.

Pattern 3 - correlate by store master, and make the action honest

// Correlate calculated → persisted on the store-assigned master, captured
// under the ORIGINAL input hash — never a re-hash of the mutated content.
$persisted = $context->createdByInputHash()[$item->inputHash()] ?? null;

if ($persisted === null) {
    // Reported on this item — never a silent `continue` that drops a cascade child.
    $result->reportUnchanged($item);
    continue;
}

$result->link($item, $persisted); // the scope row re-links to real persisted identity

And the verdict now comes from persisted identity, never a local hash:

final readonly class PersistedActionResolver implements ActionResolverInterface
{
    public function resolve(
        StateItemInterface $item,
        ?RuleSetInterface $persisted,
    ): BatchUpsertAction {
        if ($persisted === null) {
            return BatchUpsertAction::WasUnchanged; // nothing landed → never claim updated
        }

        return $persisted->isNewMaster()
            ? BatchUpsertAction::WasUpdated
            : BatchUpsertAction::WasUnchanged;
    }
}

The principle underneath all three: identity is computed by the store, and only the store. The client never re-hashes, never dedups, never decides created/updated/unchanged from a local hash. Same rules submitted from two places resolve to one store id; the verdict is desired-versus-actually-persisted identity.

The tail: symmetry, on both sides of the wire

Fixing the monolith closed the silent no-op, but left a quieter asymmetry exposed. The store hashed content without origin; the monolith hashed with it. After the honest-verdict fix, a pure re-stamp own → project correctly reported was_unchanged... but a genuine project → own markup move could still collapse into a single master, because the two sides didn't agree on whether origin was part of identity.

The resolution was symmetry, not cleverness - one field, appended on the side that was missing it:

// origin IS part of the canonical form — the client includes it, so the store
// must too. Two sets identical in patterns but differing only in origin (own vs
// project) must hash differently, else FindOrCreate collapses them and prod
// loses markup on brand→region moves.
serialized := field(r.Type, r.Match, r.Pattern, r.Priority, r.Origin)

The lesson generalizes past this bug: if a field is part of a rule set's identity for one participant, it must be part of it for every participant - or you get silent divergence no status code will ever reveal.

Testing: prove it against the real persist path, with real people in the loop

The uncomfortable truth that started everything is that the first conclusion - "there's no bug" - came from testing the wrong path. So the regression test drives the real store-persist path: RED before the fix, GREEN after. A dry_run or compute-only test would never catch it, by construction.

public function testRestampCascadesToEveryConfig(): void
{
    $this->forceStoreWriteSource();
    $this->store->willResolveDivergent();      // reproduce the prod shape in-process

    ($this->batchApply)($this->request);        // the real batch apply, real persist path

    // Assert on PERSISTED state, never the response body.
    // Before the fix this is RED: the response claims was_updated, the DB has nothing.
    $this->assertScopeLinkedToPersistedMaster('config:15');
    $this->assertMaterializedRuleCount('config:15', 16);
}

And - the part the automated suite can't do alone - real people validate the business process. A 21-case production write-matrix drove the hunt; two cases stayed red until the identity fix, then flipped green. Those cases live in a shared harness with stable, numbered case ids, so a verdict is a coordinate (case, deploy-version) rather than a vibe:

Case	before the fix	after the fix
4 - plain create	✅	✅
15 - restamp own→project	❌	✅
16 - move project→own	❌	✅
18 - shrinking REMOVE	❌	✅
22 - cascade to N configs	✅	✅

Unit tests prove the mechanism; a person running the real process proves it matters. During a migration you need the second one most, because the failures that hurt are the ones that return 200.

Three things make the automated side fast and honest: one PostgreSQL testcontainer for the whole run (not one per test - isolate at the data layer with a transaction that rolls back), a stub that reproduces the divergence deterministically, and assertions on what landed in the DB, not on the 200.

Where this goes next - the parallel run

This is a snapshot of a seam mid-migration, and the interesting question isn't "is it fixed" - it is - but "what does Stage 1 make possible." The answer is Stage 2: the cascade compute itself leaves PHP for a stateless Go service, and for a while two engines run in parallel over the same data. Here's the topology that Stage 1's identity fix unlocks:

LEVEL 1  two separate front doors — NOT connected to each other:
         PHP monolith (PHP-FPM, legacy)        Go API gateway (new service)

LEVEL 2  services:
         domain-rule-map svc                   rule-set-markup service
         identity (id+hash), its OWN DB,        new cascade engine, Go,
         a microservice FOR PERFORMANCE,        STATELESS: routes + mirrors,
         never touches the monolith DB          no DB of its own — writes the
                                                monolith DB directly

LEVEL 3  storage:
         monolith PostgreSQL DB (both engines' only write sink)  +  DRM's own DB

  Both engines resolve identity from domain-rule-map over gRPC. Both write the
  monolith DB. The identity service never touches it.

Three things about this picture are only safe because of the Stage-1 fix:

The new engine writes the monolith's DB directly. It has no store of its own yet - only the compute is moving, not the data. That's a deliberate, temporary coupling with an exit at Stage 4, not the end state. It's safe only because the service is stateless: it keeps nothing, so it can't drift from the DB it borrows.
domain-rule-map is a separate microservice for one reason: performance - one fast, immutable identity lookup that both engines share over gRPC. It owns identity in its own DB and never touches the monolith DB. If Stage 1 hadn't made identity a single honest authority, running two write engines over one database would multiply exactly the divergence this article is about.
The two front doors are independent. The gateway fronts the new service; the PHP-FPM monolith is its own legacy front door. They aren't wired together - which is what lets an automated consistency agent drive both engines and diff persisted truth, case by case, until the new engine is provably backward-compatible. (That's Part 1.5.)

Every one of those is guarded by a constraint pulled straight out of this incident: one identity model (no second hasher on any client), honest action (verdict from persisted state, never a computed hash), cascade and shrinking-remove must actually persist, and symmetric identity across the wire. A fixed bug protects one line of code. A constraint plus a regression test protects the design - so when Stage 2 runs two engines over one database, it physically cannot resurrect a ghost we already exorcised.

AI as a multiplier, and what that means for a migration

I lean hard on AI coding assistants for work like this, and my honest, strongly-held take is that AI amplifies good engineers and exposes weak ones. It's a multiplier, not a crutch.

Look back at what actually caught and fixed this bug. A teammate noticed the business output was wrong. A colleague corrected a false hypothesis with one sentence about an invariant. A real production write, watched at the persistence layer, exposed the divergence. AI made generating the code - the harness, the interfaces, the fixtures - cheap and fast. It did nothing to tell me a was_updated could be a lie; in fact it happily helped me build a plausible, tidy wrong hypothesis that a human who knew the system dismantled in a line.

Point a multiplier at a disciplined process - real design, small interfaces, deterministic tests, verification against truth, and humans validating the actual business flow - and it collapses the old "fast or correct" trade-off into fast and correct. Point it at no methodology and it produces plausible-looking wrongness faster than you can review it. A silent no-op wearing a 200 is exactly that kind of wrongness.

This is Part 1 of a series

I started here because identity split across a migration boundary is where the breakup can hurt users most quietly. The rest of the series follows the other fronts:

Part 1: A 200 OK That Saved Nothing: the parallel run - two engines over one database, and how an agent proves they agree on persisted truth.
Part 1.5 - Two rule engines, one truth: the parallel run - two engines over one database, and how an agent proves they agree on persisted truth.
Part 2 - Design trade-offs: what "identity" means when a migration splits it, and why the ownership of a hash function is an architecture decision.
Part 3 - Performance: why the cascade is leaving PHP for a stateless Go service - the timeout budget and the one-pass in-memory merge.
Part 4 - Cutover strategies: running both write paths live behind a runtime master-switch, and flipping the master safely.

If you build serious backends - Symfony, Go, or the messy space between a monolith and the services growing out of it - follow along. And if you're already running two systems that must agree on identity: how are you proving they agree - on the response, or on what actually persisted? I'd genuinely like to compare notes.

Breaking the Monolith - Part 4: Flipping the Master Live

Anton Brilliantov — Mon, 10 Aug 2026 09:59:17 +0000

How you retire a live write path without a flag day: run both paths at once, make the switch a runtime decision, prove the two systems agree on what actually persisted - before and after you flip - and only then let the switch stop being reversible, because there's nothing left to switch back to.

👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go. For a while now I've been on the kind of project that teaches you the most: carefully breaking a large PHP monolith into Go microservices while it's still very much alive and serving a real business. The first three parts of this series lived on one seam - a 200 OK that saved nothing, the design fork of who owns identity, and moving the slow cascade off PHP. This part is the payoff: how that seam actually gets cut over in production, with nobody noticing. The running notes live on my GitHub: github.com/brilliant-almazov. No hype, just the real work.

Here's the thing nobody tells you about a monolith breakup: writing the new service is the easy half. The dangerous half is the day you make it authoritative - the moment the new code, not the old, decides what lands in the database. Get that wrong and you don't get a stack trace. You get a 200 OK that saved nothing - the exact production incident that opened this series, and the exact reason everything below exists.

The migration, in stages

Stage 0  Monolith only.
         PHP computes the cascade, persists the data, owns identity implicitly.

Stage 1  Identity extracted.
         A small Go service becomes the immutable, content-addressed master of
         identity (id + hash). PHP still computes and persists the DATA.

Stage 2  Write logic extracted.
         The cascade is re-implemented in a stateless Go service that writes
         directly into the monolith's DB. Two engines run in parallel.

Stage 3  Prove they agree.            <-- THIS ARTICLE
         Drive both paths and diff PERSISTED truth, case by case, on the deploy
         that's actually running.

Stage 4  Flip, then delete.           <-- THIS ARTICLE
         Promote the new authority behind a live, reversible master-switch, then
         retire the old path - and only then move the DATA out.

Everything upstream (the identity fix, the stateless cascade, the parallel run) exists to earn Stages 3 and 4. A cutover isn't a moment; it's the disciplined walk from "two systems, one is master" to "one system, no switch left." This article is that walk.

Context: what a cutover actually is

Strip away the specifics and the domain is simple. The system stores classification rules attached to a three-level hierarchy that cascades top-down - CLIENT ⊃ PROJECT ⊃ CONFIG - so writing a rule set at a parent node fans out and recomputes every descendant's effective set. That fan-out is the expensive write path this series has been extracting from PHP, piece by piece. (The full primer is in Part 1; here I only need the shape of it.)

Now the cutover. Strip away the domain and a cutover is a single question asked on every write: which system is allowed to be right?

Before the migration, there's one answer, hard-wired: the monolith. After the migration, there's one answer, hard-wired: the new path. A cutover is the messy middle where both answers are wired in at once and something has to choose between them per request - ideally something you can change while the service is running, without a rebuild and without a deploy.

That "something" is a master-switch flag. Not a feature flag for a UI toggle - a flag that decides which of two write paths is the master, the authority whose result is the truth. Two properties make it worth the name:

Live. You flip it on the running service. No rebuild, no image push, no rolling restart.
Reversible. If the new path misbehaves under real traffic, you flip it back in seconds - not "revert the PR and wait 20 minutes for CI and a deploy."

Reversible-in-seconds is the whole game. It's the difference between a cutover and a gamble.

The anti-pattern that quietly kills reversibility

I'll name the mistake first, because it's the one I see most and it's invisible until the worst possible moment.

You build the two write paths. You wire the switch. Then, somewhere in the name of "clean production config," someone prunes the unused path out of the object graph at build time - a compile flag, a DI profile, an if BUILD_ENV == "prod" that only ever constructs the new path. The reasoning sounds responsible: don't ship dead code, don't instantiate what you won't use.

The result is a switch that can't switch. A runtime environment variable cannot flip a seam that was compiled away. The legacy path isn't "off" - it doesn't exist in the running process. So the day the new path misbehaves, your reversible cutover reveals itself as a one-way door, and your rollback plan is a redeploy of the old binary under incident pressure. That's a flag day wearing a feature-flag costume.

The rule that avoids it: always register both implementations; let a factory decide at runtime. Dead code that a switch can reach is not dead - it's your rollback. In Go, that's a factory holding both, choosing per request:

// WritePath is the seam the cutover flips. Exactly two implementations exist —
// the legacy local path and the store path. BOTH are always constructed and
// registered. The switch chooses one per request; it never chooses at build time.
type WritePath interface {
    Apply(ctx context.Context, desired RuleSet) (Persisted, error)
}

// MasterSwitch reads a runtime flag — env var, config row, control-plane value.
// The point is that StoreIsMaster can return a different answer on the very next
// request, on the same running process, with no rebuild.
type MasterSwitch interface {
    StoreIsMaster(ctx context.Context) bool
}

type WritePathFactory struct {
    legacy WritePath   // always here — this IS the rollback
    store  WritePath   // always here
    master MasterSwitch
}

// One method: Resolve. It returns which path is authoritative right now.
// No instanceof ladder, no build tag — polymorphism decides, at runtime.
func (f *WritePathFactory) Resolve(ctx context.Context) WritePath {
    if f.master.StoreIsMaster(ctx) {
        return f.store
    }
    return f.legacy
}

Note the naming discipline, because it's load-bearing across this whole series: the factory has one method, Resolve - a resolver resolves, it doesn't grow a resolveOrBuild cousin. Both paths satisfy the same interface, so the caller routes by polymorphism, never by a type switch in shared code. And it's WritePath, MasterSwitch, RuleSet - plain PascalCase, initialisms as words, ID the one exception - not WRITEPATH or HTTPClient. Small rules, but they're what let a new implementation slot in behind the same contract without anyone downstream changing a line.

Where the switch really lives - and where it scales to

The factory above chooses between two code paths inside the monolith: the legacy local write and the store-backed write from Part 1. That's the smallest instance of the cutover. But the same discipline scales up one level, to the topology the parallel run runs on - and this is the picture that has to be exactly right, because it's easy to draw wrong:

LEVEL 1  two separate front doors — NOT connected to each other:
         PHP monolith (PHP-FPM, legacy)        Go API gateway (new service)

LEVEL 2  services:
         domain-rule-map svc                   rule-set-markup service
         identity (id+hash), its OWN DB,        new cascade engine, Go,
         a microservice FOR PERFORMANCE,        STATELESS: routes + mirrors,
         never touches the monolith DB          no DB of its own — writes the
                                                monolith DB directly

LEVEL 3  storage:
         monolith PostgreSQL DB (both engines' only write sink)  +  DRM's own DB

  Both engines resolve identity from domain-rule-map over gRPC. Both write the
  monolith DB. The identity service never touches it.

Three things about this picture matter for a cutover:

The new engine writes the monolith's DB directly. It has no store of its own yet - only the compute has moved, not the data. That's a deliberate, temporary Stage-2 coupling with an exit at Stage 4, not the end state. It's safe only because the engine is stateless: it keeps nothing, so it can't drift from the DB it borrows.
domain-rule-map is a separate microservice for one reason: performance - one fast, immutable identity lookup that both engines share over gRPC. It owns identity in its own DB and never touches the monolith DB. Identity resolution is off both engines' write path to the monolith.
The two front doors are independent. The gateway fronts the new service; the PHP-FPM monolith is its own legacy front door. They aren't wired together - which is exactly what lets a consistency agent drive both engines and diff persisted truth, case by case. The companion to Part 1 tells that story in full.

Whether the switch is choosing two code paths or two whole services, the cutover discipline is identical: register both, flip live, prove agreement against the DB, promote, delete.

Proving the two paths agree - before you dare to flip

A live, reversible switch tells you that you can flip. It says nothing about whether you should. For that you need evidence that both paths produce the same truth in the database for the same input - because the failure mode from Part 1 was precisely two systems that agreed on the HTTP response and disagreed on what persisted.

So I built a harness whose only job is to answer "do these two agree?" - and to leave proof on disk. It runs one loop:

Snapshot. Capture the current persisted state of a production polygon (a safe, disposable slice of real data) - before.json. You cannot verify a change you didn't measure before.
Act. Run the write. Dry-run by default - it computes and diffs but persists nothing. Real writes are gated behind an explicit flag and are reversible, so "act for real" is a deliberate, recoverable choice, never the default.
Verify truth in the DB. This is the step Part 1 was built on: don't trust the response. Re-fetch the persisted state and diff response-versus-persisted. A 200 with was_updated means nothing until the row says so.
Restore the polygon. Leave the data exactly as found. The next run must start from the same known state, or your matrix drifts into noise.
Persist the full transcript to disk. Input, response, persisted state, diff, verdict - all of it, written out. This path has no production logs. If the harness doesn't record what happened, nothing did. The transcript is the evidence.

// The harness is one interface with one honest method. It does not "test"; it
// records a verdict against truth and leaves a transcript behind.
type AgreementCheck interface {
    Run(ctx context.Context, c Case) (Verdict, error)
}

func (h *Harness) Run(ctx context.Context, c Case) (Verdict, error) {
    before, err := h.polygon.Snapshot(ctx, c.Target)
    if err != nil {
        return Verdict{}, fmt.Errorf("snapshot: %w", err)
    }
    defer h.polygon.Restore(ctx, before) // restore always — even on failure

    resp, err := h.act.Apply(ctx, c.Input) // dry-run unless real writes are gated on
    if err != nil && !errors.Is(err, ErrDryRun) {
        return Verdict{}, fmt.Errorf("act: %w", err)
    }

    persisted, err := h.polygon.Fetch(ctx, c.Target) // TRUTH, not the response body
    if err != nil {
        return Verdict{}, fmt.Errorf("verify: %w", err)
    }

    v := Verdict{Case: c.ID, Deploy: h.deploy, Agree: resp.Matches(persisted)}
    return v, h.transcript.Persist(ctx, c, resp, persisted, v) // no logs — this is the record
}

errors.Is(err, ErrDryRun) rather than err == ErrDryRun, because the moment a decorator wraps that error the direct comparison goes silently false and the harness starts "acting for real" when it thinks it's dry - the exact class of quiet lie this series is about.

A verdict is a coordinate, not a vibe

One run of the harness produces one fact: case X agreed (or didn't) on deploy version Y. That pairing is the unit that matters. A verdict isn't "it works now" - it's a coordinate (case, deploy-version).

The cases are stable and numbered, shared verbatim with a reviewer. When they and I say "case 16," we mean the same scenario forever - the project → own markup move that stayed red in Part 1 until the identity fix landed. That stability is what turns a conversation about correctness into a conversation about a grid:

Case	`v4.5.8`	`v4.5.9`
4 - plain create	✅	✅
15 - restamp own→project	❌	✅
16 - move project→own	❌	✅
18 - shrinking REMOVE	❌	✅
22 - cascade to N configs	✅	✅

The ❌→✅ flips between two deploy tags are the safety argument for the cutover. I don't flip the master because the code looks right; I flip it because every case that used to disagree now agrees, on the deploy that's actually running, with a transcript on disk to prove it. And the reason this rigor exists at all is coordinate (any case, the day we shipped a silent no-op) - a 200 OK that persisted nothing, in production, from Part 1. You earn the right to flip a master by making disagreement a thing you can see.

The end state: when the switch stops being reversible

Here's the part that surprised me. A reversible switch is not the goal - it's scaffolding. The goal is to make it un-reversible on purpose.

The cutover progresses in stages, each guarded by a green agreement matrix on the running deploy:

Both paths live, legacy is master. The new path runs in shadow; the harness proves it agrees. Flip risk: zero, because you haven't flipped.
New path becomes master, legacy stays registered. You flip the switch live. If anything drifts, you flip back in seconds - the legacy path is still in the object graph, still your rollback. This is the reversible window, and you live in it deliberately until the matrix is boringly green across many deploys.
Identity moves fully into the store. The monolith's local copy stops being a source of truth and becomes a projection - a read-model derived from the store, not an authority that can disagree with it. There's now only one place that can be right.
The switch stops being reversible - because there's nothing to switch back to. Once the legacy path owns no truth, keeping it registered guards nothing; it just preserves a ghost. You remove it. The MasterSwitch collapses to a constant, then disappears. The seam is cut - and only then does the data migration (Stage 4 proper) begin.

That last step is how you retire an old write path without a flag day. There's no heart-stopping midnight deploy where the old world ends and the new one begins in one transaction. There's a slow, evidenced walk: register both, flip live, watch the matrix, promote the store to sole authority, and only then delete the path you no longer need. The big-bang deploy everyone fears is avoided not by being brave on the night, but by never having a night.

AI as a multiplier, and what it can't do for a cutover

I lean hard on AI coding assistants for work like this, and my honest, strongly-held take is that AI amplifies good engineers and exposes weak ones. It's a multiplier, not a crutch.

AI made the mechanics of this cutover cheap - generating the factory, the harness scaffold, the transcript serializer, the matrix formatter. Fast, tireless, genuinely useful. What it did not do - could not do - is any of the judgment the cutover actually turns on. It won't tell you that pruning the legacy path at build time silently voids your rollback. It won't insist you diff persisted state instead of the response, because the response looks authoritative and the model has no scar from a 200 that lied. It won't decide when the matrix is green enough to promote the store to sole authority - that's a risk call a human owns.

Point a multiplier at a disciplined cutover - both paths registered, a runtime switch, agreement proven against truth, verdicts as coordinates, an unhurried walk to irreversibility - and it collapses "fast or safe" into fast and safe. Point it at "the new service passes its unit tests, ship it as master" and it'll help you build a flag day faster than you can schedule the incident review.

This is Part 4 of a series

Four parts, one seam, told honestly - from the bug that exposed it to the cutover that closes it:

Part 1 - A 200 OK that saved nothing: the silent success, and why identity must live in exactly one place.
Part 1.5 - Two rule engines, one truth: the parallel run behind this cutover - old PHP engine and new stateless Go engine live at once, proven equal through the gateway.
Part 2 - Who owns a hash function: where to cut, and why ownership of identity is an architecture decision, not an implementation detail.
Part 3 - A stateless cascade in Go: why the slow cascade left PHP for a stateless Go service, and how to measure the win honestly.
Part 4 - this one: flipping the master live, proving agreement against truth, and retiring the old path without a big-bang deploy.

That's one complete seam. The pattern generalizes to every remaining seam in the monolith: extract a service, prove it agrees with the old one against persisted truth, flip a live and reversible switch, watch the matrix, promote the new authority, delete the old path. No six-month freeze. No flag-day gamble. Just good engineering, amplified - and a human deciding when it's actually safe to flip.

If you build serious backends - Symfony, Go, or the messy space between a monolith and its microservices - follow along. And if you're mid-cutover right now: can you flip your master back in ten seconds? If the honest answer is "we'd have to redeploy," you don't have a cutover yet - you have a flag day with extra steps. I'd genuinely like to compare notes.

Breaking the Monolith - Part 3: The Cascade That Timed Out

Anton Brilliantov — Thu, 06 Aug 2026 10:31:29 +0000

Why the rule-set write path is leaving PHP for a single stateless Go service - the timeout budget that forced it, the one-pass in-memory cascade that replaces thousands of database round-trips, and how to measure the win without fabricating a hero number.

👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go. For a while now I've been on the kind of project that teaches you the most: carefully breaking a large PHP monolith into Go microservices while it's still very much alive and serving a real business. The running notes live on my GitHub: github.com/brilliant-almazov. No hype, just the real work. Part 1 was about a silent 200 OK that saved nothing - the bug that taught me two systems, each with its own database, must agree on identity. Part 2 went deeper on that identity question. This part is about the reason the whole write path moved at all: it was too slow, and on the biggest writes it timed out in production.

Where this stands today: the extraction is built and deployed. The new Go engine now runs in parallel with the old PHP one - the same cascade computed by two engines at once - while an automated agent proves they agree before either is trusted. That parallel run is its own story: a companion to Part 1 covers it in full. This part is the why underneath it - the timeout that forced the extraction, and the shape of the service that replaced the slow path.

The report that started it wasn't a bug. It was a dashboard. Our error tracker kept lighting up with the same signature: the Go API gateway returning 524 on rule-set writes, clustered on the largest parent nodes. 524 is the gateway giving up - the origin took longer than its ceiling (~100s) to answer, so the connection got cut out from under it. The client retried. The retry was just as slow. Same node, over and over, a steady drip of red.

Nothing was wrong, exactly. The writes that finished were correct. They just didn't always finish.

The migration, in stages

Stage 0  Monolith only.
         PHP computes the cascade, persists the rule data, owns identity implicitly.

Stage 1  Identity extracted.          (Part 1)
         A small Go service becomes the immutable, content-addressed master of
         identity (id + hash). PHP still computes and persists the rule DATA.

Stage 2  Write logic extracted.       <-- THIS ARTICLE
         The cascade compute leaves PHP for a stateless Go service that writes
         directly into the monolith's DB. It moved because the PHP path TIMED OUT.

Stage 3  Prove they agree.            (Part 1.5)
         An agent drives both engines and diffs persisted truth, case by case.

Stage 4  Flip, then delete.
         Promote the Go engine, retire PHP, and only then move the DATA out.

A few years ago this was one PHP/Symfony monolith on PostgreSQL. We've been carving it into focused Go services one seam at a time - the old system paying the bills while each new service earns its place next to it. Part 1's seam was identity: a Go service, the rule-set store, became the immutable, content-addressed authority for "is this the same rule set or a different one?" This part is the next seam over - the write and cascade path itself - and unlike Part 1 it's a performance story, not a correctness one. The trigger was blunt: on the biggest tenants the PHP write path stopped finishing inside the gateway's budget.

Primer: what the cascade actually does

Strip away the domain and it's simple. The system stores classification rules attached to nodes in a three-level hierarchy that cascades top-down:

CLIENT            rules here apply to everything beneath
  └── PROJECT     rules here apply to every config of the project
        └── CONFIG   a specific target (search-engine × device × locale)

Writing a rule set at a parent doesn't just save that node - it fans out. Every descendant's effective rule set is recomputed by merging what it inherits from above with what it owns locally, with a clear precedence: a narrower level overrides a broader one. The rule is replace-own-keep-inherited - a node's own rules replace its previous own rules, but everything inherited from ancestors is carried down untouched. The fully-resolved result for each node is its materialized state - what a given CONFIG actually sees at request time.

The cost hides in the fan-out. A CONFIG-level write touches one node. A CLIENT-level write can touch every CONFIG under that client - realistically a few thousand of them for a big tenant. That's the write that returns 524.

Why PHP timed out: O(descendants) round-trips inside one request

The PHP cascade was written the way you'd naturally write it against a database: node by node. For each descendant it did a small read-modify-write cluster - load the node's set, merge the parent's rules in, resolve identity against the rule-set store, write the mirror. Correct, readable, and completely fine for one node. But it ran serially, inside one synchronous request, once per descendant.

For a CLIENT with a few thousand CONFIGs that's a few thousand round-trip clusters, chained - each a millisecond or three of network and query time that's invisible at N=1 and lethal at N=thousands. Wall-clock climbed linearly with subtree size until the biggest parents drifted past the gateway's ~100s ceiling. The database wasn't the bottleneck. The chattiness was - thousands of sequential trips across a wire, each waiting for the last.

You don't fix that by making PHP faster. You fix it by stopping the round-trips.

The shape of the new service

The extracted service - I'll call it the rule-set-markup service - is deliberately small. It is:

One synchronous gRPC process. One binary. No worker. No queue. No async anything. Desired state comes in on a gRPC call, the fully-persisted result goes back on the same call.
Stateless. It owns zero persistent state of its own. It machines over two things it doesn't own: the monolith's database and the rule-set store. Restart it, run N replicas, kill it between phases - all equivalent, because there's no in-process state to lose.

One consequence deserves its own sentence, because it looks like a sin until you see the stage it's in: the service has no database of its own - it writes straight into the monolith's PostgreSQL. That's deliberate. Only the compute (the cascade) is moving out right now; the rule data still belongs to the monolith, which is still the source of truth. Moving the data is a separate, later migration. Letting the new engine write the monolith's own tables means its output lands exactly where the old engine's did - which is precisely what makes "do the two engines agree?" a question you answer by reading one database. The coupling is temporary, documented, and safe only because the service is stateless: it keeps nothing, so it can't drift from the database it borrows.

The whole thing is one method behind one interface:

// The entire service surface: desired state in, persisted result out.
// One method — no per-level variants, no per-node-kind branches.
type Markup interface {
    Resolve(ctx context.Context, desired DesiredState) (Result, error)
}

Inside Resolve, four phases run in strict order: merge → cascade in memory → persist in one transaction → post-commit notify. No phase reaches for a resource a later phase hasn't earned yet.

Phase 1-2: the whole cascade, in memory, in one pass

The core move is to stop treating each node as a database trip. The service loads the entire affected subtree with one query, then does the whole cascade in memory - a single top-down walk, parent to child, carrying the inherited set down as it goes.

The cascade itself is one small interface with one method - the direct descendant of Part 1's resolve:

// A resolver resolves. One method. Inherited set + a node's own set → its
// effective set. Pure function of its inputs — no I/O, no clock, no DB.
type Resolver interface {
    Resolve(inherited, own RuleSet) RuleSet
}

// replace-own-keep-inherited: own rules win at their own level; everything
// from ancestors is carried down as-is.
type CascadeResolver struct{}

func (CascadeResolver) Resolve(inherited, own RuleSet) RuleSet {
    effective := inherited.Clone()
    effective.ReplaceOwn(own)
    return effective
}

And the one-pass walk that applies it across the loaded subtree:

// One top-down pass. Each node's resolved set becomes the inherited input for
// its children. No recursion into the database — the subtree is already in RAM.
func (m *markup) cascade(root *Node, desired DesiredState) []Resolved {
    out := make([]Resolved, 0, root.DescendantCount())
    var walk func(node *Node, inherited RuleSet)

    walk = func(node *Node, inherited RuleSet) {
        own := desired.OwnFor(node.ID) // the write; empty for untouched nodes
        effective := m.resolver.Resolve(inherited, own)
        out = append(out, Resolved{NodeID: node.ID, Set: effective})

        for _, child := range node.Children {
            walk(child, effective)
        }
    }

    walk(root, desired.InheritedInto(root.ID))
    return out
}

Round-trips for the whole cascade: one read for the subtree, regardless of how many thousand descendants it has. The identity resolution against the rule-set store is batched too - the computed sets go over in one call, not one-per-node. O(descendants) collapsed to O(1) in trips.

Phase 3: persist in one transaction - and tx is not an argument

Everything the cascade produced lands in one transaction. Either the whole fan-out commits or none of it does; there's no half-cascaded subtree to reconcile after a crash.

The part I care about stylistically: the transaction is a property of the store, never a parameter threaded through method signatures. A func Persist(ctx, tx, rows) is procedural and leaks the transaction into every caller. Instead the executor (pool or tx) is a field, and a scoped instance is minted with WithTx:

// The executor — pool or tx — is a field, never a method argument.
type Store struct {
    exec Executor
}

// WithTx returns a tx-scoped Store. Same methods, now bound to the tx.
func (s *Store) WithTx(tx pgx.Tx) *Store {
    return &Store{exec: tx}
}

// The tx lifecycle is encapsulated here. Callers never see BEGIN/COMMIT/ROLLBACK
// and never hold a tx handle — they get a scoped Store and use it.
func (s *Store) RunInTx(ctx context.Context, fn func(context.Context, *Store) error) error {
    tx, err := s.exec.Begin(ctx)
    if err != nil {
        return fmt.Errorf("begin: %w", err)
    }

    if err := fn(ctx, s.WithTx(tx)); err != nil {
        if rbErr := tx.Rollback(ctx); rbErr != nil && !errors.Is(rbErr, pgx.ErrTxClosed) {
            return errors.Join(err, rbErr)
        }
        return err
    }

    return tx.Commit(ctx)
}

So the persist phase reads as one atomic block, with no transaction plumbing visible at the call site:

err := m.store.RunInTx(ctx, func(ctx context.Context, tx *Store) error {
    return tx.PersistCascade(ctx, resolved) // one batched write, tx-scoped
})
if err != nil {
    return Result{}, fmt.Errorf("persist cascade: %w", err)
}

Error identity, everywhere, goes through errors.Is / errors.As - never err == x, never a type switch - so a wrapped error five layers down still matches. (The ErrTxClosed check above is exactly why: something upstream may already have closed the tx, and a bare == would miss it the moment anyone adds a %w.)

Phase 4: post-commit notify - after the wire is quiet

Recalculation of anything downstream is not part of this transaction. Once the commit succeeds, the service tells the domain-rule-map service which CONFIGs actually changed, and that system owns the recompute. Notify-after-commit, never inside it: the write path stays short, and a slow or unavailable consumer can never hold the write transaction open. If notification fails, the committed state is still correct - the recalc is re-derivable from what landed, not from a message we have to guarantee delivery of.

Why Go here, and not Symfony

I write plenty of the write domain in Symfony and I'd defend it. But this service isn't rich domain logic - it's fan-out and serialization: load a subtree, merge sets in a tight loop, serialize thousands of rows, write them once. That workload rewards tight tail latency and lean, predictable memory, and punishes per-request warmup and per-object overhead - exactly Go's shape. The decision wasn't "Go beats PHP." It was that this specific shape (hot, wide, serialization-bound, no deep domain rules) is what Go is for, and the domain-heavy write logic that stays in the monolith is what Symfony is for.

Statelessness is what makes the cutover safe

Here's the property that matters more than the speed. The service holds no state of its own - every input is loaded fresh from the monolith DB and the rule-set store on each call, every output written straight back. There is nothing in the process to be inconsistent with the world.

That's what makes the parallel run safe - and it's not upcoming, it's live. The old PHP path and the new Go path run side by side today, traffic flips per-request, the Go service restarts mid-migration, scales to N replicas, or gets killed between phase 2 and phase 3 - none of it can corrupt anything, because a killed call simply didn't commit and the next one recomputes the same cascade from the same source of truth. Statelessness isn't a résumé buzzword here; it's the precondition that lets me change the write path under live traffic without a flag-day gamble. (Proving the two engines actually agree, run-for-run, is its own article.)

Measuring the win honestly

I'm not going to hand you a hero number bolted to a real tenant - that's how anonymized case studies quietly leak client identity, and the shape of the win is the honest part anyway.

The unit that matters is round-trips per write, because that's what scaled with subtree size and blew the budget:

	PHP client-side cascade	Stateless Go service
DB round-trips	`O(descendants)` - one cluster per node	`O(1)` - one subtree read
Store identity calls	one per node	one batched call
Cascade	node-by-node, serial	one in-memory pass
Persist	interleaved per node	one transaction
Worst-case parent	past the gateway's ~100s ceiling → `524`	sub-second for the same subtree

The budget frame kept me honest: the gateway gives a write ~100 seconds before it returns 524. The PHP path spent that budget on network chatter - thousands of sequential trips, each mostly waiting. The Go path spends almost none of it there: one read, one in-memory pass, one write. I trust that framing because it's a structural claim - trips went from linear to constant - not a benchmark I can accidentally cherry-pick.

AI as a multiplier - the same throughline

I lean hard on AI assistants for work like this, and my honest take hasn't changed across the series: AI amplifies a good engineer and exposes a weak one. Generating this service - the interfaces, the walk, the store scaffolding, the tests - was fast. What AI did not do was tell me the problem was round-trip count rather than "PHP is slow," or that the fix was to load the subtree once rather than to parallelize the existing per-node loop (which would have kept the chattiness and just raced it). It didn't decide that the transaction belongs to the store as a field, or that notify has to live after the commit. Point the multiplier at a diagnosed problem and a designed shape and it collapses the old "fast or correct" into fast and correct. Point it at "make the cascade faster" with no diagnosis and it will cheerfully help you parallelize the wrong thing at high speed.

This is Part 3 of a series

The seam is diagnosed, designed, and built stateless on purpose. What's left is the scary part - swapping it in under live traffic.

Part 1 - A 200 OK that saved nothing. The silent 200 OK, and why identity must live in exactly one place.
Part 1.5 - Two rule engines, one truth. The service in this part is now live in parallel with the PHP one - how an agent proves the two engines agree, through the gateway, on persisted truth.
Part 2 - Who owns a hash function. What "identity" means when two systems each own a database, and why owning a hash function is an architecture decision.
Part 4 - Flipping the master live. Running the old PHP cascade and the new stateless Go service side by side behind a runtime master-switch, flipping traffic safely, and retiring the old path without a big-bang deploy. Statelessness (this part) is what makes that flip reversible - Part 4 is how you actually pull it.

If you build serious backends - Symfony, Go, or the messy space between a monolith and the services growing out of it - follow along. Concrete, code-first, honest about the mistakes.

And if you've moved a write path off a monolith: how did you measure the win without a number that fingerprints your client? I'd genuinely like to compare notes.

Breaking the Monolith - Part 2: Who Owns a Hash Function

Anton Brilliantov — Tue, 04 Aug 2026 07:55:02 +0000

Part 1 was a bug story: a 200 OK that saved nothing. This is the design story underneath it - why the ownership of a hash function stopped being an implementation detail and became an architecture decision, and how I reasoned about the trade-offs while both systems were live and one of them had just been handed the right to say "these two things are the same."

👋 Hi again, I'm Anton - a software engineer working mostly in PHP/Symfony and Go, carving a large live monolith into services one seam at a time. In Part 1 (and its expanded cut) I chased a silent no-op: the API cheerfully reported was_updated while the database saved nothing. That post was about the symptom. This one is about the decision that made the symptom possible - and the decision that fixed it for good. Running notes live on my GitHub: github.com/brilliant-almazov. Honest version, mistakes left in.

The migration, in stages

You cannot reason about "who owns identity" without knowing which stage you're standing in when you ask. A big-bang rewrite is a bet you make once and lose slowly; the strangler-fig alternative moves the seam one named stage at a time. Here is the seam this whole series lives on - the rule-set write path - with a marker on the stage this article is about:

Stage 0  Monolith only.
         PHP computes the cascade, persists the rule data, owns identity implicitly.

Stage 1  Identity extracted.          <-- THIS ARTICLE
         A small Go service becomes the immutable, content-addressed master of
         identity (id + hash), in its OWN database, answering over gRPC.
         PHP still computes and persists the rule DATA.
         The question: WHO is allowed to decide two rule sets are "the same"?

Stage 2  Write logic extracted.       (Part 1.5)
         The cascade is re-implemented in a stateless Go service that writes
         directly into the monolith's DB. Two engines run in parallel — and both
         resolve identity from the Stage-1 service over gRPC.

Stage 3  Prove they agree.
         An agent drives both engines and diffs persisted truth, case by case.

Stage 4  Flip, then delete.
         Promote the Go engine, retire PHP, and only then move the DATA out.

Part 1 was a bug that lived at Stage 1. Part 2 is the decision at Stage 1 that the bug was a violation of. Get this decision right and Stage 2 - two write engines over one database - is safe. Get it wrong and Stage 2 multiplies the divergence Part 1 was about.

Primer: what "identity" even means here

Strip away the specifics and the domain is simple. The system stores classification rules - patterns that tag web domains with a meaning ("this domain is ours", "this one is a competitor", "this one is irrelevant"). A rule is roughly {pattern, type, match-mode, priority}. A rule set is an ordered collection of those rules attached to one node in a hierarchy that cascades top-down:

CLIENT            rules here apply to everything beneath
  └── PROJECT     rules here apply to every config of the project
        └── CONFIG   a specific target (search-engine × device × locale)

A write at a parent level fans out: the engine recomputes every descendant's effective rule set, merging what it inherits from above with what it owns locally, narrower level winning.

Now the word that carries this whole article: identity. It's the answer to one question - "is this the same rule set I already have, or a different one?" You answer it with a hash function: reduce the content to a fixed fingerprint, and two sets are "the same" exactly when their fingerprints match. This is content-addressing - the same idea Git uses for commits and blobs. Same content → same hash → same object. Change one byte → new hash → new object.

Sounds like plumbing. It isn't. The instant a second system needs to agree on whether two rule sets are the same, "how do we hash content" stops being a private detail of the monolith and becomes a contract between two services. And a contract has an owner.

The design fork: where does the hash function live?

When I introduced the new Go service - I'll call it the rule-set store (the domain-rule-map service, when its identity-master role is the point) - I had exactly three places to put the authority for identity. This is the fork the whole series turns on:

                 who computes a rule set's ID?

  A  each system for itself      B  the client owns it        C  the store owns it
     ┌────────┐  ┌────────┐         ┌────────┐  ┌──────┐         ┌────────┐  ┌──────┐
     │monolith│  │ store  │         │monolith│  │ store│         │monolith│  │ store│
     │ hash() │  │ hash() │         │ hash() │  │ bytes│         │ content│  │ hash()│
     └───┬────┘  └───┬────┘         └───┬────┘  └──┬───┘         └───┬────┘  └──┬───┘
         │  DISAGREE │                  │  owns    │                 │  asks   │
         └────┬──────┘                  └───→ id   ▼                 └───→ id   ▼
       two answers = a bug        migration never moves       ONE owner of identity
         ✗ REJECTED (a law)         ✗ REJECTED (owns nothing)     ✓ CHOSEN

Option A - every system hashes for itself. The monolith computes its hash, the store computes its own, each trusts its own answer. This is the one I accidentally shipped, and Part 1 is the crater. Two hash functions is one hash function too many: the moment they disagree by a single field, you get silent divergence that no status code will ever reveal. Rejected - not as a preference, as a law.

Option B - the client owns identity, the store is a dumb blob bucket. The store just persists bytes under a key the monolith picks. Simple, but it means the monolith stays the authority forever - the migration makes no progress, because the thing I'm trying to extract (the decision "these are the same") never actually leaves PHP. Rejected: it's a service that doesn't own anything.

Option C - the store owns identity, everyone asks it. The store is the single place that decides what a rule set's id is. The monolith computes content; the store computes identity, in its own database, and answers over gRPC. Chosen - and everything else in this article is a consequence of that choice.

Consequence 1: the store is deliberately, aggressively dumb

Because the store owns identity and only identity, I could make it the simplest service on the platform. It is immutable and content-addressed: you hand it a canonical rule set, it returns the one master for that content, creating it only if absent. It never mutates, never cascades, never validates business rules. It creates exactly what you give it.

// The store owns identity and nothing else. Hand it a canonical rule set,
// it returns the single master for that content — creating one only if absent.
// It lives in its OWN database and is reached over gRPC; it never touches the
// monolith's DB.
type RuleSetStore interface {
    FindOrCreate(ctx context.Context, set CanonicalRuleSet) (RuleSetID, error)
}

func (s CanonicalRuleSet) Hash() Hash {
    h := sha256.New()
    for _, r := range s.Rules { // rules already in canonical order
        io.WriteString(h, field(r.Type, r.Match, r.Pattern, r.Priority, r.Origin))
    }
    return Hash(h.Sum(nil))
}

The trade-off I accepted on purpose: the store is dumb, so the data and the cascade logic stay in the monolith. It can't answer "what rules does config 15 actually see" - that's not its job. A less disciplined design would have let the store grow a little validation here, a little cascade there, until it became a second monolith. Keeping it a small, immutable, content-addressed authority is precisely what makes it easy to reason about - and easy to trust as the one owner of identity.

There's a second reason it's a separate service and not a library: performance. Identity resolution is one fast, immutable lookup that both engines - the PHP monolith today, the stateless Go cascade at Stage 2 - share over gRPC. A shared, cache-friendly authority is worth a network hop; a library copied into two runtimes is Option A wearing a disguise.

Consequence 2: create is not recalculate - and conflating them is a category error

This is the distinction a teammate corrected me on in one sentence, and it's structural, not cosmetic.

Creation is one transaction: send the computed set to the store, get back the master ids, and save the monolith's local mirror - all or nothing.

Recalculation is a different operation entirely: a post-commit fan-out over everything that changed, recomputing descendants' effective rule sets. It runs after the create transaction has committed, because it reacts to what actually landed.

I don't want that boundary living in a comment or an inline runInTransaction(fn () => …) at the call site - I want it to be a type. The write handler knows only the domain; a transactional decorator runs it in exactly one transaction; an after-commit decorator schedules the recompute through a port the transaction dispatches only once it commits. Nothing in the handler ever sees BEGIN/COMMIT or calls an afterCommit():

// The domain handler — zero transaction awareness, zero commit-hook awareness.
interface WriteHandler
{
    public function handle(RuleSetWrite $write): WriteResult;
}

final readonly class CreateRuleSet implements WriteHandler
{
    public function __construct(
        private RuleSetStore $store,
        private MirrorProjection $mirror,
    ) {}

    public function handle(RuleSetWrite $write): WriteResult
    {
        $masters = $this->store->createAll($write->computed); // identity — the store decides
        $this->mirror->save($write->computed, $masters);      // local projection of that identity

        return new WriteResult($masters, $write->changed);
    }
}

// Creation runs INSIDE one transaction — a decorator, not an inline closure at the call site.
final readonly class Transactional implements WriteHandler
{
    public function __construct(
        private WriteHandler $inner,
        private TransactionManager $tx,
    ) {}

    public function handle(RuleSetWrite $write): WriteResult
    {
        return $this->tx->run(fn (): WriteResult => $this->inner->handle($write));
    }
}

// Recalculation is a SEPARATE, post-commit concern — also a decorator. It schedules the
// recompute through an AfterCommit port that the transaction fires only after COMMIT.
final readonly class RecalcAfterCommit implements WriteHandler
{
    public function __construct(
        private WriteHandler $inner,
        private AfterCommit $afterCommit,
    ) {}

    public function handle(RuleSetWrite $write): WriteResult
    {
        $result = $this->inner->handle($write);
        $this->afterCommit->schedule(fn () => $result->changed->recompute());

        return $result;
    }
}

// The boundary is now a TYPE, not a comment: schedule inside the tx, dispatch after commit.
$write = new Transactional(
    new RecalcAfterCommit(new CreateRuleSet($store, $mirror), $afterCommit),
    $tx,
);

Why does this matter for ownership? Because the tempting shortcut is to treat the store's echo of a create as the source of truth for content - to read identity back and re-derive what changed from it. That's the category error. The store answers "what is this set's id"; it does not answer "what did this write change downstream." Recalculation owns the second question, and it owns it after commit. Blur the two and you get exactly Part 1's ghost: a create that reports success while the recalc it implied never happened.

Consequence 3: dual authority over data is a trade-off; dual authority over identity is a bug

Here's the uncomfortable part. During the transition the monolith still keeps a local mirror of every rule set, and a runtime master-switch flag chooses the write path per request: store path or legacy path.

$writeSource = $this->masterSwitch->current(); // WriteSource::Store | WriteSource::Legacy

So for a while, two places hold rule-set data. That looks like exactly the two-authorities trap I just spent three sections rejecting. It isn't - and the distinction is the whole point of this section:

     DATA                                    IDENTITY
     ┌─────────────┐   ┌─────────────┐        ┌─────────────┐
     │ store       │   │ monolith    │        │   store     │  ← the ONLY authority
     │ (own DB)    │   │ mirror      │        │  (own DB)   │    that decides an id
     │ id + hash   │   │ (monolith   │        └──────┬──────┘
     └─────────────┘   │  DB)        │               │ FindOrCreate over gRPC
        two data           projection               │
        stores, kept       of the id the      monolith ── asks ──┘
        coherent           store handed it     mirror ── never decides, only records
     ── a TRADE-OFF ──                          ── a LAW ──

The mirror is a projection, not a second opinion. It never computes an id; it only records the id the store handed it. The legacy path still writes the mirror the old way, which is why the mirror has to exist at all during cutover - but nothing on the mirror is ever allowed to decide identity. The client never re-hashes, never dedups, never rules created/updated/unchanged from a local hash. The one moment I let the mirror's hash cast a verdict is the exact moment Part 1 blew up.

Note the storage picture this implies, because it's a fact people get wrong: the identity store keeps identity in its own database and is reached over gRPC - it is not squatting in the monolith's schema. The rule data (and the mirror) live in the monolith's PostgreSQL DB. So there really are two databases here, but only one of them is ever allowed to answer the identity question.

The cost of the data trade-off is real: two data stores to keep coherent, a flag to reason about, a window where a bug can hide in the seam. I took it deliberately because the alternative - freeze writes, migrate identity, thaw - means downtime a live business won't grant me. Dual authority over data is the price of a zero-downtime cutover. Dual authority over identity is a bug. Knowing which is which is the whole job.

Consequence 4: symmetry is a law, not a nicety

Once identity has one owner, a second law falls out of it. In Part 1, after the honest-verdict fix, a quieter asymmetry was still exposed: the store hashed content without the origin field; the monolith hashed with it. Two participants, two different notions of "same". Result: a genuine project → own markup move could still collapse two distinct sets into one master, and prod would silently lose markup.

The fix wasn't cleverness - it was symmetry. One field, appended on the side that was missing it (you can see r.Origin already in the Hash() above):

If a field is part of a rule set's identity for one participant, it must be part of it for every participant - or you get silent divergence no status code will ever reveal.

That's the general law. But symmetry-by-copying-a-field is the tactical fix; it leaves two hashers that I now have to keep in lockstep forever. The end state is cleaner and it's where this seam is heading: stop storing origin as sticky data on a rule at all, and instead derive it from where the rule sits in the cascade at read time. A rule's level would then follow from structure, not from a label that can drift out of sync between two services. Derived identity can't diverge, because there's only one thing to compute it from. That's the difference between patching a symptom and dissolving a bug class.

The discipline that keeps this legible: naming and interfaces

None of the above survives contact with a codebase unless the code reads like the design. A few rules I refuse to bend, because they're what let a new implementation slot in without a rewrite:

A resolver has exactly one method: resolve. Not resolve, resolveFor, executed, executedFor - one verb, one contract. The fact that now matters (what the store actually persisted) enters as a nullable argument, not as a parallel method or an interface zoo.
Everything behind an interface; instanceof ladders in shared code are banned. The runner routes by polymorphism, so adding a write path never touches the orchestrator.
final readonly classes, declare(strict_types=1) everywhere. Immutable value objects can't drift after construction - the same discipline the store enforces at the service level, enforced again at the object level.

declare(strict_types=1);

interface ActionResolverInterface
{
    public function resolve(
        StateItemInterface $item,
        ?RuleSetInterface $persisted,
    ): BatchUpsertAction;
}

final readonly class PersistedActionResolver implements ActionResolverInterface
{
    public function resolve(
        StateItemInterface $item,
        ?RuleSetInterface $persisted,
    ): BatchUpsertAction {
        // Binary by nature: a newly minted master → updated; anything else
        // (nothing landed, or an existing master reused) → unchanged.
        return $persisted?->isNewMaster()
            ? BatchUpsertAction::WasUpdated
            : BatchUpsertAction::WasUnchanged;
    }
}

On the Go side the same instinct shows up in naming: identifiers read as ordinary words - HttpClient, RuleSetID, HeadSha - never shouted as HTTPClient or HeadSHA. The only initialism that stays uppercase is ID. It's a small rule, but consistent naming is how a reader trusts that resolve means resolve everywhere, and that identity means the store everywhere.

Where this goes next - the parallel run

This decision - one owner of identity, in its own DB, reachable over gRPC - is what makes the next stage safe. At Stage 2 the cascade compute itself leaves PHP for a stateless Go service, and for a while two write engines run in parallel over the monolith's database. Here's the topology the identity decision unlocks:

LEVEL 1  two separate front doors — NOT connected to each other:
         PHP monolith (PHP-FPM, legacy)        Go API gateway (new service)

LEVEL 2  services:
         domain-rule-map svc                   rule-set-markup service
         identity (id+hash), its OWN DB,        new cascade engine, Go,
         a microservice FOR PERFORMANCE,        STATELESS: routes + mirrors,
         never touches the monolith DB          no DB of its own — writes the
                                                monolith DB directly

LEVEL 3  storage:
         monolith PostgreSQL DB (both engines' only write sink)  +  DRM's own DB

  Both engines resolve identity from domain-rule-map over gRPC. Both write the
  monolith DB. The identity service never touches it.

Three things about this picture are only safe because the identity decision was made honestly:

The new engine writes the monolith's DB directly. It has no store of its own yet - only the compute is moving, not the data. That's a deliberate, temporary coupling with an exit at Stage 4, not the end state. It's safe only because the service is stateless: it keeps nothing, so it can't drift from the DB it borrows.
domain-rule-map is a separate microservice for one reason: performance - one fast, immutable identity lookup that both engines share over gRPC, in its own DB, never touching the monolith DB. If identity weren't a single honest authority, running two write engines over one database would multiply exactly the divergence Part 1 was about.
The two front doors are independent. The gateway fronts the new service; the PHP-FPM monolith is its own legacy front door. They aren't wired together - which is what lets an automated consistency agent drive both engines and diff persisted truth, case by case, until the new engine is provably backward-compatible.

That's Part 1.5.

AI as a multiplier, again - because this is where it bites

I lean hard on AI coding assistants for work like this, and the throughline from Part 1 holds: AI amplifies a good engineer and exposes a weak one. Nowhere is that sharper than in a design decision like "who owns the hash function." An assistant will happily generate Option A - each system hashing for itself - because locally, in each file, it looks completely reasonable. Two tidy Hash() methods, both correct in isolation, that together form a silent-corruption generator. The tool produces plausible-looking code at high speed; it does not tell you that identity must have exactly one owner, or that create and recalc are different operations, or that a field in one hash must be in the other. That judgment came from a person who knew the invariant and a teammate who corrected me in a sentence.

Point the multiplier at a real design - one identity owner, a dumb immutable store, an explicit create/recalc boundary, symmetry as a law, single-method interfaces - and it collapses "fast or correct" into fast and correct: I get the boilerplate, the value objects, the tests for free, and I spend my time on the decisions that don't have a locally-obvious answer. Point it at no design and it generates the crater faster than you can review it. That's the difference between amplifying engineering and automating its absence.

This is Part 2 of a series

Part 1 was the bug; Part 2 was the decision under it - that ownership of identity is an architecture choice, not plumbing. The seam keeps moving:

Part 1 - A 200 OK that saved nothing. The silent no-op this decision fixes, and its expanded cut with the full staged-migration map.
Part 1.5 - Two rule engines, one truth. The cascade now runs as two live services - the PHP engine and a new stateless Go one - writing the same database in parallel, proven equal through the gateway before either is trusted.
Part 3 - A stateless cascade in Go. Why the write/cascade path is leaving PHP for a stateless Go service: the timeout budget forcing it, the one-pass in-memory merge, and how to measure the win honestly.
Part 4 - Flipping the master live. Running both write paths behind the runtime master-switch and flipping the master safely - retiring the legacy path and the mirror without a big-bang deploy, until identity lives fully in the store and the switch has nothing left to switch back to.

If you build serious backends - Symfony, Go, or the messy space between a monolith and the services growing out of it - follow along. Concrete, code-first, honest about the mistakes.

And if you're mid-migration with two systems that must agree on identity: which one owns it? If your answer is "both", that's not two answers - it's a bug waiting for a business-side human to notice. I'd genuinely like to compare notes.

Breaking the Monolith - Part 1.5: Two Rule Engines, One Truth

Anton Brilliantov — Fri, 31 Jul 2026 11:48:40 +0000

👋 Hi, I'm Anton - a software engineer working mostly in PHP/Symfony and Go. This is a companion to Part 1 of a series about carefully breaking a large PHP monolith into Go microservices while it keeps serving a real business. Part 1 was a bug story: an API that answered was_updated while the database saved nothing. This part is what came after the fix - the phase where the write logic is being lifted out of PHP into a new service, and for a while both the old engine and the new one are live at the same time. Running notes live on my GitHub: github.com/brilliant-almazov. No hype, just the real work.

There's a moment in a monolith breakup that the tutorials skip. It's not writing the new service - that's the fun part. It's the stretch where the new service is done, deployed, and running next to the old one, and both compute the same answer for real production writes, and your entire job is to prove that they never, ever disagree. Two engines. One truth. That's this article.

The migration, in stages

I'll keep coming back to a staged picture, because the whole point of a strangler-fig migration is that it happens in steps you can name, not in one heroic deploy. Here is the seam this series lives on - the rule-set write path - laid out as stages:

Stage 0  Monolith only.
         PHP computes the cascade, persists rule data, owns identity implicitly.
         This is where the big parent writes timed out in prod (Part 3).

Stage 1  Identity extracted.
         A small Go service - the domain-rule-map service - becomes the
         immutable, content-addressed master of identity (id + hash).
         PHP still computes and persists the rule data. (Part 1 & 2.)

Stage 2  Write logic extracted.  <-- WE ARE HERE
         The cascade is re-implemented in a stateless Go service - the
         rule-set-markup service. But the monolith still owns the rule DATA,
         so the new service writes DIRECTLY into the monolith's database.
         Same cascade logic now lives in TWO engines: PHP and Go, in parallel.

Stage 3  Prove they agree.       <-- AND HERE
         An automated agent drives BOTH engines through the Go API gateway
         and diffs the persisted result, case by case, until the new engine
         is provably backward-compatible with the old one.

Stage 4  Flip, then delete.  (future)
         Promote the Go engine to master, retire the PHP path, and only later
         move the DATA itself out of the monolith. The duplication ends here.

The rest of this article is Stages 2 and 3 - the awkward, interesting middle where the system runs two rule engines on purpose.

Primer: what the two engines actually do

Strip away the domain and it's simple. The system stores classification rules - patterns that tag web domains ("this one is ours", "this one is a competitor", "this one is irrelevant"). Rules attach to nodes in a three-level hierarchy that cascades top-down:

CLIENT            rules here apply to everything beneath
  └── PROJECT     rules here apply to every config of the project
        └── CONFIG   a specific target (search-engine × device × locale)

A write at a parent doesn't just save that node - it fans out, recomputing every descendant's effective rule set by merging what it inherits from above with what it owns locally. The fully-resolved result per node is its materialized state - what a given config actually sees at request time.

That fan-out is the logic being duplicated. In Stage 0 it was a PHP loop, node by node, one database round-trip cluster per descendant - correct, and on the biggest parents, slow enough to time out (that's Part 3's whole story). In Stage 2 it's a stateless Go service that loads the subtree once and does the whole cascade in memory in a single pass. Same meaning, two implementations, running side by side.

Why duplicate the logic at all?

The honest answer: because you cannot safely delete the old engine until you've proven the new one agrees with it, and you cannot prove agreement without running both. Duplication isn't the accident here - it's the method. The alternative is a flag day: swap PHP for Go in one deploy and find out in production whether they match. On a live business, that's not a migration, it's a bet.

So for the length of Stage 2-3, the same cascade is computed by two engines:

The legacy engine - the PHP cascade inside the monolith. Mature, correct, and slow on big writes.
The new engine - the stateless rule-set-markup service in Go. Fast, and unproven until the agreement matrix says otherwise.

A runtime master-switch decides which engine's result is authoritative per request (that's Part 4's mechanism). But authoritative-or-not, both can run - the new one in shadow, computing and persisting-or-diffing without being trusted yet. The duplication is the cost of admission for a reversible cutover.

The part that surprises people: the new service writes into the monolith's database

Here's the design decision that gets a raised eyebrow every time I explain it, so let me be blunt about it.

The new Go service does not have its own database. It writes directly into the monolith's PostgreSQL database.

Not through the monolith's API. Not into a store of its own that gets synced back. It opens a connection to the same Postgres the PHP monolith owns, and writes the scope links and rule rows there, in one transaction.

Why on earth would a "microservice" reach straight into another service's database - the textbook cardinal sin? Because of what stage we're in. The monolith is still the monolith: it is still the source of truth for rule data. Moving the data out is a separate, later, riskier migration (Stage 4). What's moving now is only the compute - the cascade logic - not the storage. Decoupling those two migrations is deliberate:

LEVEL 1  two separate front doors — NOT connected to each other:

    ┌────────────────────┐                    ┌────────────────────────┐
    │  PHP monolith      │                    │  Go API gateway        │
    │  (PHP-FPM)         │                    │  front door for the    │
    │  legacy front door │                    │  new service           │
    └────────────────────┘                    └────────────────────────┘
             │  │                                   │ routes
      writes │  │ gRPC                              ▼
             │  │ id+hash            ┌────────────────────────────────┐
             │  │                    │  rule-set-markup service       │
             │  │                    │  new cascade · Go · STATELESS  │
             │  │                    │  routes + mirrors · no DB yet  │
             │  │                    └────────────────────────────────┘
             │  │                          │ writes directly   │ gRPC id+hash
LEVEL 2      │  └──────────┐               │        ┌──────────┘
             │             ▼               │        ▼
             │   ┌────────────────────────┐│   (same gRPC identity call)
             │   │  domain-rule-map svc   ││
             │   │  identity (id+hash)    ││
             │   │  a microservice, FOR   ││
             │   │  PERFORMANCE — never   ││
             │   │  touches the monolith  ││
             │   │  DB · its OWN DB       ││
             │   └────────────────────────┘│
LEVEL 3      │                             │
             ▼                             ▼
    ┌─────────────────────────────────────────────────────────────────┐
    │                   monolith PostgreSQL DB                         │
    │          owns the rule DATA — both engines' write sink           │
    └─────────────────────────────────────────────────────────────────┘

The rule of the seam is: migrate one thing at a time. Extract the identity decision first (Stage 1, its own service, its own store). Extract the cascade compute next (Stage 2, a stateless service that borrows the monolith's DB). Extract the data ownership last (Stage 4). If I tried to move compute and data in one step, I'd be debugging a new algorithm and a new storage layer at the same time, with no way to tell which one broke a case. By letting the new engine write the monolith's own tables, its output lands in exactly the place the old engine wrote to - so "do they agree?" becomes a question I can answer by reading one database, not by reconciling two.

The statelessness is what makes this safe rather than horrifying. The new service holds no state of its own - every input is read fresh from the monolith DB and the domain-rule-map service on each call, every output written straight back. There is nothing in the service to be inconsistent with the database it borrows, because it keeps nothing. Restart it, run N replicas, kill it between phases: a killed call simply didn't commit, and the next call recomputes the same cascade from the same source of truth. Borrowing another service's database is a sin when the borrower is stateful and drifts; it's a controlled, temporary coupling when the borrower keeps nothing and the coupling is written down as a stage with an exit.

Concretely, the new engine is a thin stateless layer on the hot write path: it routes each request, mirrors the cascade into the monolith's own tables, and resolves identity from the domain-rule-map service over gRPC. That last point matters and the diagram makes it explicit - the domain-rule-map service never touches the monolith's database. It owns identity (id + hash) in its own store, and it's a separate microservice for one reason: performance - a single fast, immutable identity lookup that both engines share. Both the legacy PHP monolith and the new Go engine ask it for identity over gRPC and then write the resolved rows into the monolith DB themselves; the identity service is on neither engine's write path to that DB. And because the markup service sits inline on every write, its latency budget is tight - "stateless" here also means "nothing to warm up, nothing to fsync, nothing between the request and the answer."

Stage 3: an agent that proves the two engines agree

Two engines writing the same tables is only safe if something is relentlessly checking that they produce the same truth. That something is an automated consistency agent, and it is the most important piece of the whole parallel run.

Its job is narrow and it does not "test" in the unit-test sense. It runs the real business flow through the real front door and records a verdict against the database. One loop, per case:

Enter through the gateway. Every check goes through the Go API gateway, not by calling a service directly. This matters: backward compatibility is a promise about what a client observes through the front door, not about internal wiring. If the answer through the gateway is identical, the swap is invisible to everyone who matters - which is the entire definition of backward compatible.
Snapshot the polygon. Capture the current persisted state of a production polygon - a safe, disposable slice of real data - as before. You cannot verify a change you didn't measure first.
Run the same input through both engines. Same request, legacy engine and new engine. Dry-run by default: it computes and diffs but persists nothing. Real writes are gated behind an explicit flag and are reversible.
Diff the persisted truth, not the response. This is the lesson Part 1 was built on: don't trust the 200. Re-fetch what actually landed in the monolith DB from each engine and compare those. A response that says was_updated means nothing until the row agrees.
Restore the polygon. Leave the data exactly as found, so the next case starts from the same known state.
Write the transcript to disk. Input, both responses, both persisted states, the diff, the verdict. This path has no production logs - if the agent doesn't record what happened, nothing did.

The verdict is not "it works". It's a coordinate: (case, engine, deploy-version) -> agrees | differs. The cases are stable and numbered, so when a reviewer and I say "case 16" we mean the same scenario forever - the project -> own markup move that stayed red in Part 1 until the identity fix landed.

// The consistency agent runs one case end to end and records a verdict.
// It does not assert in a test runner; it diffs persisted truth and leaves
// a transcript. Everything enters through the gateway - the real front door.
type ConsistencyAgent interface {
    Check(ctx context.Context, c Case) (Verdict, error)
}

func (a *Agent) Check(ctx context.Context, c Case) (Verdict, error) {
    before, err := a.polygon.Snapshot(ctx, c.Target)
    if err != nil {
        return Verdict{}, fmt.Errorf("snapshot: %w", err)
    }
    defer a.polygon.Restore(ctx, before) // restore always, even on failure

    // Same input, both engines, through the gateway. Dry-run unless gated.
    legacy, err := a.gateway.Apply(ctx, c.Input, EngineLegacy)
    if err != nil && !errors.Is(err, ErrDryRun) {
        return Verdict{}, fmt.Errorf("legacy apply: %w", err)
    }
    fresh, err := a.gateway.Apply(ctx, c.Input, EngineMarkup)
    if err != nil && !errors.Is(err, ErrDryRun) {
        return Verdict{}, fmt.Errorf("markup apply: %w", err)
    }

    // TRUTH, not the response body: read what each engine persisted.
    legacyState := a.polygon.Fetch(ctx, c.Target, legacy)
    freshState := a.polygon.Fetch(ctx, c.Target, fresh)

    v := Verdict{
        Case:   c.ID,
        Deploy: a.deploy,
        Agrees: legacyState.Equal(freshState), // persisted-vs-persisted
    }
    return v, a.transcript.Persist(ctx, c, legacy, fresh, legacyState, freshState, v)
}

errors.Is(err, ErrDryRun) rather than err == ErrDryRun, because the moment a decorator wraps that error a bare comparison goes silently false and the agent starts acting for real when it thinks it's dry - the exact class of quiet lie this whole series is about.

A verdict is a grid, and the grid is the backward-compatibility argument

One run produces one fact: case X agreed (or didn't) between the two engines on deploy Y. Stack those facts and you get the only artifact that lets me flip an engine to master with a straight face:

Case	legacy vs new `v4.6.0`	legacy vs new `v4.6.1`
4 - plain create	✅	✅
15 - restamp own→project	❌	✅
16 - move project→own	❌	✅
18 - shrinking REMOVE	❌	✅
22 - cascade to N configs	✅	✅

The ❌→✅ flips between two deploy tags are the compatibility proof. I don't promote the new engine because the code looks right; I promote it because every case that used to differ now agrees, on the deploy that's actually running, with a transcript on disk. Backward compatibility stops being a hope and becomes a grid you can point at.

And the grid earns its keep in the ugly direction too. When the new engine diverges on a case, the diff is specific - this config, these rules, this field - because both engines wrote the same tables and I'm comparing rows, not vibes. A divergence is a bug report with coordinates, not a "sometimes it's off" ticket.

The honest edges of running two engines

I'd be lying if I sold duplication as free. The costs are real and worth naming:

Drift. Two implementations of one algorithm will drift unless something forces them together. The agreement agent is that force - but it only catches what's in the case matrix. A behavior nobody wrote a case for can diverge silently. The mitigation is that the matrix grew out of real production incidents (Part 1's cases came from a 21-case write-matrix), so it covers the shapes that actually bite.
A shared database is a coupling with a deadline. The new engine writing the monolith's tables is a temporary, documented coupling, not a pattern to copy. It has an exit (Stage 4). The danger is letting "temporary" become "forever" - so the coupling is written into the stage plan with the step that removes it, not left as tribal knowledge.
Two write paths is two chances to be wrong. The parallel run doubles the surface where a bug can hide, for the duration. That's the price of not doing a flag day. You buy reversibility with complexity, and you pay it back the moment you delete the legacy path in Stage 4.

None of these is a reason not to run in parallel. They're the reasons the consistency agent is not optional.

AI as a multiplier, and what it can't do for a parallel run

I lean hard on AI coding assistants for work like this, and my take hasn't shifted across the series: AI amplifies a good engineer and exposes a weak one. It's a multiplier, not a crutch.

AI made the mechanics of the parallel run cheap - generating the second engine's scaffolding, the agent loop, the snapshot/restore, the transcript serializer, the matrix formatter. Fast and genuinely useful. What it did not do is any of the judgment the parallel run turns on. It won't tell you to migrate compute before data, or that the new service borrowing the monolith's database is safe only because it's stateless and the coupling has an exit. It won't insist you diff persisted rows instead of the response, because the response looks authoritative and a model has no scar from a 200 that lied. It won't decide when the grid is green enough to promote an engine - that's a risk call a human owns. Point a multiplier at a disciplined parallel run - staged migration, stateless borrowing, agreement proven against persisted truth through the front door - and it collapses "fast or safe" into fast and safe. Point it at "the new service passes its tests, cut over" and it'll help you build a flag day faster than you can schedule the incident review.

This is a companion to Part 1

Part 1 was one bug across two code paths. This part is the same discipline scaled up to two whole services: prove they agree on persisted truth before you trust either one. The seam keeps moving:

Part 1 - A 200 OK that saved nothing: the silent no-op, and why identity must live in exactly one place.
Part 2 - Who owns identity: why the ownership of a hash function is an architecture decision.
Part 3 - The cascade that timed out: why the write engine left PHP for a stateless Go service - the extraction this article runs in parallel with.
Part 4 - Flipping the master: the runtime switch that decides which engine is authoritative, and how the parallel run ends.

If you build serious backends - Symfony, Go, or the messy space between a monolith and the services growing out of it - follow along. And if you're mid-extraction right now with old and new code computing the same thing: how are you proving they agree - on the response, or on what actually persisted? I'd genuinely like to compare notes.