DEV Community

Pavol Z. Kutaj
Pavol Z. Kutaj

Posted on

Explaining CDC (Change Data Capture)

def

  • around for 10-15 years or so but what is new is the demand for change
  • CDC is the technology that turns a database change log into a stream of events
  • kind of like a time series data, but with the change that happens to the database itself
  • every single write every, single insert update captured in the change log of the database
  • we have something that reads the database log
  • not the database directly — we're not querying the tables and using database resources that way
  • we're looking at the log file of the database
  • turning that into a stream of events that then you can do stuff with

use

  • backup and disaster recovery
  • you can create exact point in time replicas of the database
  • this that batch data loading misses is— changes that happen in between the frequencies of the snapshots right so if you're trying to do something like fraud detection or you're
  • trying to detect or train a machine learning model on on sort of real world data sets and data
  • things can happen at a higher frequency than your snapshot in the database and you'd miss that with change data capture you get all of that and there's tons of

AWS Security LIVE!

Join us for AWS Security LIVE!

Discover the future of cloud security. Tune in live for trends, tips, and solutions from AWS and AWS Partners.

Learn More

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay