What Is Change Data Capture (CDC) and How It Works on Google Cloud

Keeping analytics pipelines real‑time and resource‑efficient is table stakes in 2025.

That’s where Change Data Capture (CDC) shines—streaming only the rows that changed instead of bulk‑copying entire tables.

In this post, you’ll learn:

1. What CDC is (and why it matters).
2. Three core implementation patterns—query‑based, trigger‑based, log‑based.
3. How to pick the right Google Cloud tool—Datastream or a DIY Debezium + Dataflow combo.
4. A quick‑reference exam tip for anyone chasing the Google Cloud Professional Data Engineer cert.

---

🔁 What Exactly Is CDC?

Change Data Capture (CDC) = detecting inserts, updates, and deletes in a source database and pushing just those deltas downstream (e.g., into BigQuery).

Result:

• No heavy full‑table copies.
• Near–real‑time dashboards and ML features.
• Lower source‑DB load.

---

🔍 CDC Patterns Every Engineer Should Know

1. Query‑Based CDC (Timestamp / Version Column)

SELECT *
FROM customers
WHERE last_updated_timestamp > '2025-07-12 21:20:00';

✅ Pros	❌ Cons
Dead‑simple scripting	Misses deletes
No extra tooling	Adds query load
	Requires schema change (extra column)

---

2. Trigger‑Based CDC

Flow: Triggers (AFTER INSERT/UPDATE/DELETE) copy changes into a *_history table.

✅ Pros	❌ Cons
Captures all ops, incl. deletes	High write‑time overhead
Built‑in audit trail	Harder to maintain at scale

---

3. Log‑Based CDC (Modern Standard)

Reads the DB’s transaction log (MySQL binlog, Postgres WAL, etc.).

✅ Pros	❌ Cons
Low latency (near real time)	Needs specialized tool
Minimal DB impact	Setup can be tricky
Captures deletes & schema changes

---

🚀 Implementing CDC on Google Cloud

Option A — Datastream (Managed, Serverless)

• Log‑based CDC for MySQL, Postgres, Oracle, AlloyDB.
• Streams raw events into BigQuery staging tables.
• Auto‑executes MERGE so target tables stay current.
• Handles schema drift for you.

Perfect when you want “set it and forget it” replication.

Option B — Debezium + Pub/Sub + Dataflow (DIY Flex)

1. Debezium connectors tail the transaction log.
2. Changes land in Pub/Sub.
3. Dataflow applies custom transforms → BigQuery.

Use this path when you need complex, in‑flight transformations or to support a niche source DB Debezium already speaks.

---

🎓 Exam Tip

For the Google Cloud PDE exam, default to Datastream for relational‑to‑BigQuery CDC.
Reach for Dataflow + Debezium only if the scenario explicitly calls for heavy transformations or bespoke routing.

---

🧠 Pattern Cheat‑Sheet

Pattern	Captures Deletes?	Source DB Load	Complexity	GCP Tool of Choice
Query‑Based	❌	🔺 Medium	🟢 Low	N/A (custom script)
Trigger‑Based	✅	🔺 High	🔺 Medium	N/A (DB triggers)
Log‑Based	✅	🟢 Low	🔺 High	Datastream, Debezium

---

Ready to Build?

CDC turns stale ETL batches into streaming insights with surprisingly little effort—especially with Datastream doing the heavy lifting.

Have questions, war stories, or tips? Drop them below—let’s level‑up together 💬

---

Thanks for reading! If you found this helpful, consider following me for more posts on data engineering, GCP, and real‑world pipeline design.