DEV Community

Cover image for 🔥 Two Services, One Table, Zero Isolation
Kyryl
Kyryl

Posted on

🔥 Two Services, One Table, Zero Isolation

Two services writing to the same table is a distributed monolith wearing a microservice badge. You split the codebase, you kept the database, and the split you actually shipped is cosmetic. Every cost of the split is real. The isolation you were supposed to get for that cost is not.

The decision that does not feel like an architecture decision

Nobody sits in a design review and votes to build a distributed monolith. It happens one convenience call at a time. Team A owns orders, team B needs order data for its own service, and the fastest way to get it is to point B's code at the same Postgres instance and the same orders table. No new endpoint to build, no contract to agree on, no deploy to coordinate. It ships this sprint instead of next quarter.

That decision gets filed under data access. It is not a data-access decision. It is the architectural boundary, decided by default, by whoever wrote the first query. The table has one schema, and from that moment both services are load-bearing on that schema staying exactly the way it is. Nobody wrote that dependency down anywhere. It lives in two codebases that do not import each other and cannot see each other's queries.

What the coupling looks like when it breaks

Here is the incident, not hypothetically, the way it actually shows up.

Service A owns the table by every reasonable definition, it was there first, its team migrated the schema originally. A needs to ship a feature. The feature needs a new required field, so someone adds:

ALTER TABLE orders ADD COLUMN fulfillment_channel VARCHAR(32) NOT NULL DEFAULT 'standard';
Enter fullscreen mode Exit fullscreen mode

A's migration tooling runs it, A's tests pass, because A's tests only ever insert through A's own code path, which now always sets fulfillment_channel. A's deploy goes out clean. Nobody on A's team thinks about service B, because as far as A's team is concerned, B is not part of this change. A does not know B writes to this table at all, or if it does, nobody flagged this migration for review.

Service B writes to orders directly, from its own repository, using its own hand-rolled insert statement that predates the new column entirely. Two things can happen next, and both are bad.

If the column had no default, B's inserts start failing immediately with a constraint violation. That is the good outcome, loud and fast, a page goes off, someone finds the cause within the hour.

The worse outcome is what actually happened above: a default value. B's inserts keep succeeding. Every row B writes silently gets fulfillment_channel = 'standard', whether or not that is true. No error. No failed health check. No alert. The data is just wrong, quietly, for every order B touches, from the moment A's migration landed until someone downstream notices the fulfillment numbers do not add up. That could be hours. It is more often days, and by the time someone in the incident review asks "wait, since when has B been writing that channel," the wrong data is already in reports, already fed into whatever dashboard finance trusts.

Nobody connects the two events, the migration and the corrupted writes, because nothing in either codebase points at the other. The only shared artifact is the table itself, and tables do not send review requests.

Why splitting the codebase did not split the coupling

The instinct once you are burned by this is to blame the migration, add a review step, require anyone touching a shared table to ping the other team first. That is a process patch on an architecture problem, and it will hold until someone forgets, or a new hire does not know the informal rule exists, or the "other team" has been renamed twice since the rule was written down in a wiki nobody reads.

The actual problem is that two services with independent deploys, independent on-call, independent codebases are still, underneath all of that, one system with one shared piece of mutable state. A network call between them would at least force a versioned contract, something with a schema of its own that changes deliberately, gets reviewed, gets a deprecation window. A shared table has no such contract. Its schema is the contract, enforced by nothing except discipline, and discipline is the thing that fails first under deadline pressure.

You paid for the split. Two deploy pipelines, two on-call rotations, a network hop between the two halves whenever one calls the other for anything else. What you did not get, in exchange for that cost, is the one thing the split was supposed to buy: a boundary strong enough that one team's change cannot silently break the other. That is what "distributed monolith" means in practice. Every operational cost of microservices, none of the isolation.

The honest trade-off

There are two real fixes, and neither one is free.

The first: pick one service as the actual owner of the table, and every other consumer goes through that owner's API. Whatever queries B was running directly against orders become HTTP or gRPC calls to A. A's schema changes are now A's problem to manage behind a versioned interface, and B is protected from anything A does internally as long as the contract holds. The cost is real: you have to build that API surface, migrate every direct query B has, and accept the latency and availability coupling that comes with a synchronous call replacing a local join.

The second: actually split the data. B gets its own table, its own copy of whatever fields it needs, kept in sync through an event stream, A publishes an OrderUpdated event whenever the row changes, B consumes it and updates its own copy. Now B genuinely does not depend on A's schema, A can rename columns internally all day without touching B. What you buy instead is eventual consistency: B's copy is sometimes a few seconds or minutes behind, and now you own a second, denormalized dataset that can drift from the source if the event pipeline ever drops a message. Reconciliation becomes its own ongoing job.

Neither of these is a migration you do over a sprint. Both are projects: mapping every access pattern the "other" service currently has against the shared table, deciding what actually needs to move, coordinating a cutover that does not lose writes in the transition. It is slower and more expensive than the shared table ever was, which is exactly why the shared table happened in the first place. It was the fast option. It is just not the option that gives you the isolation you were charging yourself for.

How many "microservices" in your own systems still share write access to a table, and has anyone actually mapped what breaks if either side changes the schema?

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen

The silent-default example is exactly why shared writes are the dangerous boundary: the database preserves validity while corrupting meaning.

A practical escape path can be made more incremental than “rewrite everything, then cut over.” First inventory real consumers from grants, query logs, and pg_stat_statements; give each service a distinct DB role; name one owner; then revoke non-owner DDL and eventually DML. Put the replacement API or outbox/CDC stream beside the existing path, shadow-read/compare it, backfill with a durable high-water mark, reconcile counts and business invariants, and use a fencing point so the old writer cannot resume after cutover.

The new boundary also needs explicit contracts. An HTTP endpoint is not automatically versioned, and an event stream is not automatically decoupled. Add compatibility tests/schema registry, idempotent consumers, replay/backfill procedures, and lag/drift SLOs. For fields like fulfillment_channel, an invariant or provenance field can make “defaulted because the old writer omitted it” observable during migration. Database-enforced roles turn the intended ownership diagram into something the system can actually refuse to violate.