DEV Community: Parag Paralikar

Why do we even need change-data-capture to begin with?

Parag Paralikar — Sun, 21 Sep 2025 08:37:04 +0000

CDC is a nifty tech that sniffs out changes in your database and broadcasts them like a nosy neighbor yelling over the fence. "Hey, someone just updated the inventory!" But do we really need it? Spoiler: Sometimes yes, sometimes it's overkill. Let's break it down step by step, starting from the basics, because who doesn't love a good origin story? We'll keep it light – no one wants to read a tech article that feels like chewing on dry toast.

The Simple Life: A Three-Tier REST Service That Doesn't Need CDC

Picture this: You've got a classic three-tier setup. It's like the peanut butter and jelly sandwich of software architecture – straightforward, reliable, and everyone knows how it works. At the top, there's the presentation layer (your snazzy frontend app, maybe built with React or whatever's trendy this week). In the middle, the application layer (your backend REST service, probably in Java or Node.js, serving up endpoints like a diner slinging burgers). And at the bottom? The data layer – an Oracle database, humming away like a loyal fridge full of your data goodies.

Here's how it flows:

User hits an endpoint: "POST /orders" to create a new order.

The service validates, processes, and writes to the Oracle DB: "INSERT INTO orders VALUES (stuff);"

Need to read? "GET /orders/123" – service queries the DB: "SELECT * FROM orders WHERE id=123;" and sends it back.

No drama, no fuss. Everything's synchronous, self-contained, and the service owns the whole shebang. Why no CDC? Because the service is the gatekeeper. It knows every read and write intimately – like a helicopter parent at a kids' playdate. If something changes, the service handles it right there. CDC would be like hiring a private detective to watch your own house: expensive and unnecessary. Save your money for better things, like upgrading to that premium coffee subscription.

Leveling Up: CQRS – Still No CDC Required, But Now With Events!

Okay, now let's crank it up a notch. Enter Command Query Responsibility Segregation (CQRS). Sounds fancy, right? It's basically splitting your system into two personalities: the "doer" (commands) and the "knower" (queries). Think of it as separating your messy kitchen (where you cook up changes) from your tidy living room (where you just admire the view).

In a CQRS setup:

The Command Service handles writes: User says "Update my profile!" It processes the command, writes to the Oracle DB, and then fires off an event like "ProfileUpdatedEvent" to a message queue (Kafka, RabbitMQ – pick your poison).

The Query Service subscribes to those events, updates its own read-optimized view (maybe a separate DB or cache), and serves reads super-fast: "GET /profile" pulls from the optimized store, not hammering the main DB.

Events flow directly from command to query service – no middleman needed. It's like passing notes in class: quick, direct, and if done right, the teacher (your DB) doesn't even notice the extra chatter. Why skip CDC here? Because you're in control of the code. You can bake event publishing right into the command logic. It's all in-house, no external meddling. Adding CDC would be like installing a smoke alarm in a room that's already on fire watch – redundant, and probably just annoying with false alarms.

But wait, there's a punchline: CQRS makes your system scalable and snappy, but it's still assuming you own everything. What if life's not that simple? Enter the real world...

The Legacy Nightmare: Where CDC Saves the Day (Because You Can't Touch the Code)

Now we're in the trenches – the legacy system swamp. Imagine an ancient Oracle DB that's been around longer than some developers' careers. Multiple services are dumping data into it like a potluck dinner: Your app writes orders, a billing system adds invoices, HR tosses in employee updates. And on the reading side? A circus of consumers: Your analytics dashboard pulls reports, a third-party CRM syncs customer data, maybe even a vendor's inventory tool peeks in.

The kicker? Some of these readers and writers aren't yours. They're owned by third parties – think external partners or ancient monoliths from acquisitions. You can't just stroll in and refactor their code to publish events. "Hey, random vendor, mind adding Kafka support to your 1990s Perl script?" Yeah, that'll go over like a lead balloon.

This is where CDC struts in like a superhero in a cape made of redo logs. CDC tools (like Debezium or Oracle GoldenGate) monitor the DB's change logs – those sneaky records of every insert, update, delete – and stream them out as events. No need to touch the legacy code; it's all passive listening.

Why is this the only way?

Can't modify sources: Third-party writers won't let you add event hooks. CDC eavesdrops without asking permission.

Multiple consumers: Everyone gets the changes in real-time, without polling the DB like a needy ex checking your status.

Decoupling magic: Your modern services can subscribe to these events, building fresh views or integrations, while the old guard chugs along unchanged.

Without CDC, you'd be stuck with clunky polling (querying the DB every few seconds – inefficient and laggy, like checking your fridge repeatedly for snacks that aren't there) or triggers (which can bloat the DB and cause performance hiccups). CDC keeps things clean, efficient, and drama-free. It's the duct tape holding legacy worlds together – not pretty, but boy, does it work.

Wrapping It Up: CDC – Not Always Needed, But a Lifesaver When It Is

So, why bother with CDC? In simple setups or controlled CQRS environments, you don't – it's like bringing a bazooka to a water gun fight. But in the wild west of legacy systems with untouchable third-party integrations, it's your golden ticket to sanity. It turns database whispers into actionable shouts, without rewriting the world.

If you've battled legacy beasts or have your own CDC horror stories, drop a comment below. Who knows, maybe we'll laugh about it over virtual coffee. Until next time, keep your data flowing and your code clean!

Why I'm Building Yet Another Change-Data-Capture Platform: Because Why Not Scratch That Itch?

Parag Paralikar — Sun, 14 Sep 2025 12:48:23 +0000

If you're anything like me, you've probably scrolled through endless lists of tools and frameworks, wondering, "Do we really need another one?" Well, buckle up, because today I'm kicking off a new blog series where I'm building a change-data-capture (CDC) platform for Oracle databases—right out in the open. Think of it as "build in public" meets "mad scientist in the garage." I'll share the highs, the lows, the "aha!" moments, and probably a few "why did I think this was a good idea?" confessions along the way.

Now, I can already hear the collective groan: "There are so many CDC tools out there already! Some are open-source, and—miracle of miracles—a handful even work as advertised!" Fair point. So why on earth am I adding to the pile? Simple:

"Intellectual curiosity is like an itch that demands to be scratched—until it turns into a full-blown adventure." —Me, channeling my inner philosopher while staring at code at 2 AM.

We've all devoured those shiny system design blogs, binge-watched YouTube tutorials, and powered through online courses that promise to make us architecture wizards. After two decades (give or take) in the software trenches, one universal truth stands out like a sore thumb: There are no perfect systems! Nope, not even close. Even in the fanciest Fortune 500 companies, you'll stumble upon legacy beasts lurking in the shadows—systems that chug along doing things that would make a fresh CS grad weep.

These relics are riddled with anti-patterns that feel like outright betrayals of everything we hold sacred in software engineering. But hey, real life isn't a textbook; it's a chaotic mix of tight deadlines, budget squeezes, and "it worked fine yesterday" logic. And that's exactly why we end up inventing tools that, at first glance, seem redundant. Enter change-data-capture: the unsung hero that whispers, "Hey, something just changed in the database—want to know about it?"

In an ideal world, if you've got a single, well-behaved write service handling all updates to your database, that service should just shout the changes over an enterprise event bus for everyone else to hear. Easy peasy, right? But reality loves to throw curveballs. What if you've got two write services? Or three? Or—heaven forbid—a whole circus of them, some from third-party vendors where you couldn't tweak the code if your life depended on it? Suddenly, you're playing whack-a-mole with codebases, trying to patch in event capture everywhere. It's exhausting, inefficient, and about as fun as debugging someone else's regex.

That's when you pivot to the source of truth: the database itself. Track the changes where they're born, and voila—problem (mostly) solved. But here's the plot twist: many of these legacy setups adore good old relational databases, and Oracle? Oh, Oracle reigns supreme as the king of the RDBMS jungle. It's battle-tested, robust, and... notoriously tricky to wrangle when it comes to CDC. Sure, Oracle offers fancy proprietary solutions like GoldenGate or XStream, but if you're a bootstrapping startup or a solo dev hustling from your home office, those licenses might as well be priced in unicorn tears. Out of reach!

So, what's a curious coder to do? Roll up our sleeves and build an open-source, enterprise-grade, massively scalable CDC platform tailored for Oracle (and hey, if the stars align and I don't run out of coffee, maybe extend it to other databases too). It's a bit like deciding to bake your own bread because the store-bought stuff just doesn't hit the spot—except this bread has to feed a data-hungry army without crumbling under pressure.

In this series, I'll be your guide through the entire journey. We'll dive into design decisions (why this architecture over that one?), framework picks (spoiler: no silver bullets here), library choices (the good, the bad, and the "why did I import that?"), design patterns (because who doesn't love a solid observer or two?), and the inevitable trade-offs—functional feats versus non-functional necessities like scalability, reliability, and that ever-elusive "it just works" magic.

Expect laughs along the way—maybe at my expense when I hit a wall (or a particularly stubborn Oracle quirk). I'll share code snippets, diagrams that look like they were drawn by a caffeinated squirrel, and real talk about what goes wrong (because it will). Who knows? You might even pick up a tip or two for your own projects.

So, grab your favorite beverage, subscribe if you're feeling adventurous, and let's turn this itch into something epic. First up: Scoping out the basics. Stay tuned— we're just getting started!