DEV Community: Java Only

How We Split a Legacy Monolith Into Microservices Without a Single Outage

Java Only — Mon, 20 Jul 2026 15:48:34 +0000

8 min read · telecom provisioning platform, 30M+ subscribers

Most companies avoid migrating their monolith for one reason: they imagine it as a single, terrifying event — months of a feature freeze, a weekend cutover, and a rollback plan that's really just hope. That fear is reasonable. A big-bang rewrite of a live system serving 30M+ subscribers really would be terrifying.

So we didn't do that. We used a pattern that lets you migrate a monolith one slice at a time, while the system keeps running and the product team keeps shipping features — the same pattern Martin Fowler named Strangler Fig, after the vine that grows around a host tree, gradually taking over, until eventually the original tree is no longer needed.

Why the "just rewrite it" instinct is usually wrong

The instinct to rewrite a legacy monolith from scratch is understandable — the old code is scary, undocumented, and nobody wants to touch it. But a full rewrite has a well-known failure pattern: it takes far longer than estimated, the business can't freeze feature development for that long, and by the time the rewrite is "done," the old system has changed underneath it and the rewrite is already out of date.

The alternative isn't "don't migrate." It's: migrate in slices small enough that each one is boring, and never require the business to stop shipping while you do it.

The pattern: a facade, and one slice at a time

Strangler Fig works by putting a routing layer — a facade or API gateway — in front of the monolith. At first, 100% of traffic passes through to the old system untouched. Then, one capability at a time:

Build the new version of that one capability as an independent service.
Update the facade to route just that capability's traffic to the new service.
Run both in parallel long enough to trust the new one (see "shadow traffic" below).
Retire that piece of the old monolith. Repeat for the next capability.

At every point in this process, the system is fully functional. There's no "half-migrated and broken" state — there's only "some capabilities on the new architecture, some still on the old one," which is a perfectly stable place to be for months if needed.

Picking what to peel off first

This is the decision that determines whether the whole migration feels safe or risky: start with the slice that's high-value but low-risk, not the scariest, most business-critical piece. A good first slice is:

Reasonably self-contained (doesn't touch five other modules)
Has a clear, testable boundary (you can tell if the new version is behaving correctly)
Matters enough that finishing it proves the pattern works — but isn't the piece that takes down the whole business if something goes wrong

Getting one real slice fully migrated — end to end, safely — does more for organizational confidence than any planning document. After the first slice, the second one is always easier, because now there's a proven playbook instead of a theory.

The detail most teams skip: an anti-corruption layer

When the new service needs to talk to parts of the monolith that haven't been migrated yet, it's tempting to just reach into the old database directly. Don't. That recreates the exact coupling you're trying to get away from, and now the "new" service is just as entangled as the old one.

Instead, use an anti-corruption layer — a thin translation layer that lets the new service talk to the monolith through a clean interface, without either side needing to understand the other's internal model. It's a bit more upfront work. It's the difference between a migration that gets easier over time and one that gets more tangled with every slice.

How to know it's safe to cut over: shadow traffic

Before fully switching a capability to the new service, run both versions in parallel: real production traffic hits the old path (which still returns the actual response), while the same traffic is also sent to the new service in "shadow" mode — its output is logged and compared, but never returned to the user. If the new service produces the same result as the old one across thousands of real requests, you've earned the confidence to cut over for real. If it doesn't, you find out with zero customer impact.

This is the single biggest reason this approach avoids outages: you're never trusting a new implementation on faith. You're trusting it because you already watched it agree with the old one thousands of times.

The cutover itself should be the least interesting part of the entire migration — because by the time you do it, you already know what's going to happen.

Sequencing: what actually happened, wave by wave

In practice, this played out as a small number of clearly-scoped waves rather than one continuous effort:

Wave 0 — foundations. Facade/gateway in place, observability wired up so we could actually see what "correct" looked like before changing anything.
Wave 1 — first slice. The self-contained, provable capability. Shadow traffic, then cutover. This is where the team builds real confidence, not theoretical confidence.
Wave 2+ — the rest, in priority order. Each subsequent slice reuses the same playbook: build, shadow, compare, cut over, retire the old path.
Final step — decommission. Once every capability has moved, the facade has nothing left to route to the old monolith, and it's retired for good.

Nowhere in that sequence is there a step called "freeze feature development while we rewrite everything." The product team kept shipping the entire time, because from their point of view, nothing was frozen — just steadily, quietly getting replaced underneath them.

What I'd tell someone starting this today

Your first slice is a confidence-building exercise, not just a technical one. Pick one that can actually succeed cleanly.
Never let a new service reach directly into the old system's data. The anti-corruption layer is what keeps the migration from re-creating the same mess.
Shadow traffic isn't optional if you want zero-downtime confidence. Watching the new path agree with the old one, on real traffic, is what makes cutover boring instead of scary.
A migration that requires freezing the product roadmap is a sign the slices are too big. Break it down further.

The business case for this isn't just "safer engineering." It's that your team never has to choose between modernizing the system and shipping the roadmap — because with this pattern, you were never actually choosing between the two.

Full post with more detail: [https://thinh-cloud.netlify.app/blog/strangler-fig-migration]
Book a 20-min call: https://cal.com/thinh-le-jvm/20mins

We fixed a hidden bug in a system processing 10M+ messages a day.

Java Only — Sun, 19 Jul 2026 16:03:31 +0000

9 min read · warehouse execution system, global retail & manufacturing

A few years ago I worked on a warehouse execution system (WES) — the software that tells a warehouse what to pick, pack, and ship next, in real time, for retail and manufacturing clients moving somewhere between 5 and 10 million requests a day. Orders come in, inventory gets allocated, pick tasks get generated, shipments get confirmed — and every one of those steps talks to the others through messages.

The system ran on ActiveMQ. It had worked for years. But at that volume, two problems had stopped being rare edge cases and started being a weekly occurrence: duplicate order processing, and failures nobody noticed until a customer did. This post is about what we actually changed — not just "we moved to NATS," but the specific patterns that made the new system trustworthy instead of just newer.

The problem wasn't really the broker

It's tempting to blame ActiveMQ itself. That's not quite fair. The real problem was what had been built around it over the years: a direct, synchronous-feeling coupling between "write to the database" and "publish a message," with no shared transaction between the two. At low volume, that gap almost never showed up. At 5–10M messages a day, it showed up constantly, in two specific ways:

Dual-write inconsistency. A service would commit an order update to the database, then publish an event. If the process crashed between those two steps — or the publish failed silently — the database and the message stream disagreed about what had happened. Sometimes an event fired twice for the same order. Sometimes it didn't fire at all.
No systematic way to detect "this already happened." Consumers processed whatever arrived. If a message got redelivered — which happens by design in most messaging systems, because "at-least-once" is the realistic guarantee, not "exactly-once" — the consumer had no way to know it was seeing the same order for the second time.

Put together, that's how you get an order picked twice, or a shipment confirmation that silently vanishes. Nobody had done anything "wrong" — the system had just been built for a lower-stakes version of itself, and it hadn't been revisited as the stakes went up.

Why NATS, specifically

NATS JetStream wasn't chosen because ActiveMQ is "bad" — it was chosen because the operational model fit this specific problem better: lighter-weight persistence, simpler clustering, and a consumer model that made explicit acknowledgment and replay straightforward to reason about. That mattered less than what we built on top of it — but it's worth saying plainly: the broker swap was the easy 20% of this project. The patterns below were the other 80%.

Pattern 1 — Outbox, so the database and the message stream can't disagree

The fix for dual-write inconsistency is the Outbox pattern: instead of writing to the database and then publishing a message as two separate operations, you write the business change and a row representing "an event needs to be published" in the same database transaction. A separate relay process reads unpublished rows and publishes them, then marks them published.

CREATE TABLE outbox_event (
  id            UUID PRIMARY KEY,
  aggregate_id  VARCHAR(64) NOT NULL,
  event_type    VARCHAR(64) NOT NULL,
  payload       JSONB NOT NULL,
  created_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
  published_at  TIMESTAMPTZ
);

This sounds almost too simple to matter. It matters a lot. It moves the "did this actually happen" question from "check two systems and hope they agree" to "check one row in one transaction." That single change removed an entire category of incident.

Pattern 2 — Idempotent receivers, because redelivery is normal, not a bug

Once you accept that messages can and will be delivered more than once — which is the honest default for almost any distributed messaging system — the consumer has to be able to answer "have I already handled this?" before doing anything with side effects.

@Transactional
public void handle(OrderEvent event) {
    if (processedEventRepo.exists(event.getEventId())) {
        return; // already handled — safe to ignore
    }
    applyOrderUpdate(event);
    processedEventRepo.markProcessed(event.getEventId());
}

The mechanics here are almost boring — a table of processed event IDs, checked before any state change, updated in the same transaction as the change itself. What isn't boring is how much it changes the character of incidents: instead of "why did this order get picked twice," the failure mode becomes "a message was redelivered and correctly ignored," which is invisible to the business and shows up as a log line instead of a support ticket.

Pattern 3 — Saga, for the things no single transaction can cover

Order processing touched inventory, picking, and shipment — three services, three databases, no single transaction spanning all of them. That's exactly the situation the Saga pattern exists for: model the multi-step process as a sequence of local transactions, each with a defined compensating action if a later step fails.

Concretely: if inventory allocation succeeds but pick-task generation fails, the saga triggers a compensating "release allocation" step rather than leaving the system in a half-committed state. None of this is exotic — it's the standard answer to distributed consistency without two-phase commit — but it only works if step 1 (idempotency) and step 2 (Outbox) are already solid. Saga compensations that fire against a system that can't tell "processed" from "not yet processed" just create a second category of duplicate work.

Pattern 4 — Retries with backoff, and a dead-letter queue with an actual runbook

The last piece was making failure explicit and bounded, instead of infinite or silent.

consumer:
  durable_name: order-service
  ack_policy: explicit
  max_deliver: 5
  backoff: ["1s", "5s", "30s", "2m", "10m"]
  dead_letter_subject: "orders.dlq"

A message gets a bounded number of attempts with increasing backoff. If it still fails, it goes to a dead-letter subject — and this is the part that's easy to skip and expensive to skip: a dead-letter queue that nobody monitors is just a slower, quieter way of losing messages. We paired the DLQ with an actual on-call runbook: what triage looks like, who gets paged, and what "safe to replay" versus "needs a human" looks like for each event type.

Before → After, in short:

Before: App writes to DB, then separately publishes to ActiveMQ (two uncoordinated steps) → consumer processes whatever arrives, with no way to detect a repeat → failures are silent until someone downstream notices.
After: App writes to DB and an Outbox row in one transaction → a relay publishes to NATS JetStream → an idempotent consumer checks a processed-events table before writing → failures beyond the retry budget go to a dead-letter subject with an actual runbook attached.

(There's a diagram of this in the full post on my site — link at the bottom.)

Migrating without a big-bang cutover

None of this shipped as one release. The order that mattered:

Outbox first, on the existing ActiveMQ setup. This alone fixed the dual-write problem before touching the broker at all, and proved the pattern under real load.
Idempotent receivers next, also still on ActiveMQ. This removed the duplicate-processing risk independently of the broker migration.
NATS introduced consumer-by-consumer, running in parallel with ActiveMQ for a transition window, comparing outputs before cutting each consumer over.
ActiveMQ retired last, once every consumer had been running clean on NATS for long enough to trust it.

Doing it in that order meant every step was independently safe to roll back, and the riskiest-sounding part of the project — "replace the message broker" — ended up being the calmest step, because the parts that actually caused incidents had already been fixed before the broker changed at all.

What I'd tell someone starting this today

Idempotency isn't optional at this point of the process — it's the thing that turns "redelivery" from an incident into a no-op.
Fix the dual-write problem before you touch the broker. Outbox is boring, unglamorous, and it's the highest-leverage change on this list.
A dead-letter queue without a runbook is a to-do list nobody reads. Decide who gets paged and what "replay" means before you need it at 2am.
Migrate the parts that reduce risk before the part that looks risky. The broker swap was the least stressful part of this project — because by the time it happened, the system could already tell the difference between "new" and "already handled."

If your system has a similar shape — high message volume, a broker that's shown its age, or a nagging feeling that "duplicate" incidents keep coming back in slightly different clothes — this is exactly the kind of thing a short audit is good for: not a rewrite, just an honest map of where the Outbox, idempotency, and retry gaps actually are before you touch anything.

Full post with architecture diagram: Link post
*Book a 20-min call: Book a call