Databases are critical for any application. Data constantly gets updated, inserted, and deleted. In most of cases it is important for the business to keep track of these changes due to security concerns, auditing requirements, and to keep other relevant systems up to date.
Change Data Capture (CDC) has gained popularity, precisely to address this problem. CDC is a technique used to track all changes in a database and capture them in destination systems. Debezium is a popular CDC tool that leverages database logs as the source of truth, and streams the changes to Kafka and compatible systems like Redpanda.
The key thing to note here is that while databases are typically mutable, the changes captured by the CDC system should be immutable. Only then users have a system of record that can be trusted. Parseable is built for immutable logs and streams of events and data. It is a great fit for CDC targets. The way we designed Parseable allows for long term retention and querying of logs, and it also provides a rich set of tools for analysis and monitoring.
In this tutorial, we use Redpanda, PostgreSQL, and Debezium to ingest CDC events into Parseable.
Read more in the full tutorial to:
- Set up Docker Compose
- Set up Debezium Connect
- Set up Parseable
- Set up Redpanda Connect
- Test the CDC pipeline
Top comments (1)
Great work compiling this, Jen. About to check out the full tutorial!