DEV Community

Pavol Z. Kutaj
Pavol Z. Kutaj

Posted on

Explaining CDC (Change Data Capture)

def

  • around for 10-15 years or so but what is new is the demand for change
  • CDC is the technology that turns a database change log into a stream of events
  • kind of like a time series data, but with the change that happens to the database itself
  • every single write every, single insert update captured in the change log of the database
  • we have something that reads the database log
  • not the database directly — we're not querying the tables and using database resources that way
  • we're looking at the log file of the database
  • turning that into a stream of events that then you can do stuff with

use

  • backup and disaster recovery
  • you can create exact point in time replicas of the database
  • this that batch data loading misses is— changes that happen in between the frequencies of the snapshots right so if you're trying to do something like fraud detection or you're
  • trying to detect or train a machine learning model on on sort of real world data sets and data
  • things can happen at a higher frequency than your snapshot in the database and you'd miss that with change data capture you get all of that and there's tons of

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Explore a sea of insights with this enlightening post, highly esteemed within the nurturing DEV Community. Coders of all stripes are invited to participate and contribute to our shared knowledge.

Expressing gratitude with a simple "thank you" can make a big impact. Leave your thanks in the comments!

On DEV, exchanging ideas smooths our way and strengthens our community bonds. Found this useful? A quick note of thanks to the author can mean a lot.

Okay