DEV Community

Cover image for Why do we even need change-data-capture to begin with?
Parag Paralikar
Parag Paralikar

Posted on

Why do we even need change-data-capture to begin with?

CDC is a nifty tech that sniffs out changes in your database and broadcasts them like a nosy neighbor yelling over the fence. "Hey, someone just updated the inventory!" But do we really need it? Spoiler: Sometimes yes, sometimes it's overkill. Let's break it down step by step, starting from the basics, because who doesn't love a good origin story? We'll keep it light – no one wants to read a tech article that feels like chewing on dry toast.

The Simple Life: A Three-Tier REST Service That Doesn't Need CDC

Picture this: You've got a classic three-tier setup. It's like the peanut butter and jelly sandwich of software architecture – straightforward, reliable, and everyone knows how it works. At the top, there's the presentation layer (your snazzy frontend app, maybe built with React or whatever's trendy this week). In the middle, the application layer (your backend REST service, probably in Java or Node.js, serving up endpoints like a diner slinging burgers). And at the bottom? The data layer – an Oracle database, humming away like a loyal fridge full of your data goodies.

Here's how it flows:

User hits an endpoint: "POST /orders" to create a new order.

The service validates, processes, and writes to the Oracle DB: "INSERT INTO orders VALUES (stuff);"

Need to read? "GET /orders/123" – service queries the DB: "SELECT * FROM orders WHERE id=123;" and sends it back.

No drama, no fuss. Everything's synchronous, self-contained, and the service owns the whole shebang. Why no CDC? Because the service is the gatekeeper. It knows every read and write intimately – like a helicopter parent at a kids' playdate. If something changes, the service handles it right there. CDC would be like hiring a private detective to watch your own house: expensive and unnecessary. Save your money for better things, like upgrading to that premium coffee subscription.

Leveling Up: CQRS – Still No CDC Required, But Now With Events!

Okay, now let's crank it up a notch. Enter Command Query Responsibility Segregation (CQRS). Sounds fancy, right? It's basically splitting your system into two personalities: the "doer" (commands) and the "knower" (queries). Think of it as separating your messy kitchen (where you cook up changes) from your tidy living room (where you just admire the view).

In a CQRS setup:

The Command Service handles writes: User says "Update my profile!" It processes the command, writes to the Oracle DB, and then fires off an event like "ProfileUpdatedEvent" to a message queue (Kafka, RabbitMQ – pick your poison).

The Query Service subscribes to those events, updates its own read-optimized view (maybe a separate DB or cache), and serves reads super-fast: "GET /profile" pulls from the optimized store, not hammering the main DB.

Events flow directly from command to query service – no middleman needed. It's like passing notes in class: quick, direct, and if done right, the teacher (your DB) doesn't even notice the extra chatter. Why skip CDC here? Because you're in control of the code. You can bake event publishing right into the command logic. It's all in-house, no external meddling. Adding CDC would be like installing a smoke alarm in a room that's already on fire watch – redundant, and probably just annoying with false alarms.

But wait, there's a punchline: CQRS makes your system scalable and snappy, but it's still assuming you own everything. What if life's not that simple? Enter the real world...

The Legacy Nightmare: Where CDC Saves the Day (Because You Can't Touch the Code)

Now we're in the trenches – the legacy system swamp. Imagine an ancient Oracle DB that's been around longer than some developers' careers. Multiple services are dumping data into it like a potluck dinner: Your app writes orders, a billing system adds invoices, HR tosses in employee updates. And on the reading side? A circus of consumers: Your analytics dashboard pulls reports, a third-party CRM syncs customer data, maybe even a vendor's inventory tool peeks in.

The kicker? Some of these readers and writers aren't yours. They're owned by third parties – think external partners or ancient monoliths from acquisitions. You can't just stroll in and refactor their code to publish events. "Hey, random vendor, mind adding Kafka support to your 1990s Perl script?" Yeah, that'll go over like a lead balloon.

This is where CDC struts in like a superhero in a cape made of redo logs. CDC tools (like Debezium or Oracle GoldenGate) monitor the DB's change logs – those sneaky records of every insert, update, delete – and stream them out as events. No need to touch the legacy code; it's all passive listening.

Why is this the only way?

Can't modify sources: Third-party writers won't let you add event hooks. CDC eavesdrops without asking permission.

Multiple consumers: Everyone gets the changes in real-time, without polling the DB like a needy ex checking your status.

Decoupling magic: Your modern services can subscribe to these events, building fresh views or integrations, while the old guard chugs along unchanged.

Without CDC, you'd be stuck with clunky polling (querying the DB every few seconds – inefficient and laggy, like checking your fridge repeatedly for snacks that aren't there) or triggers (which can bloat the DB and cause performance hiccups). CDC keeps things clean, efficient, and drama-free. It's the duct tape holding legacy worlds together – not pretty, but boy, does it work.

Wrapping It Up: CDC – Not Always Needed, But a Lifesaver When It Is

So, why bother with CDC? In simple setups or controlled CQRS environments, you don't – it's like bringing a bazooka to a water gun fight. But in the wild west of legacy systems with untouchable third-party integrations, it's your golden ticket to sanity. It turns database whispers into actionable shouts, without rewriting the world.

If you've battled legacy beasts or have your own CDC horror stories, drop a comment below. Who knows, maybe we'll laugh about it over virtual coffee. Until next time, keep your data flowing and your code clean!

Top comments (0)