DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 970,177 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Robin Moffatt for Confluent

Posted on • Updated on

Kafka Connect JDBC Sink: tips & tricks - video walkthrough

Using ksqlDB you can enrich streams of data, and write the resulting Kafka topic to a database. You can also build stateful aggregations and write these to a database.

πŸŽ₯ https://rmoff.dev/ksqldb-jdbc-sink-video

Alt Text

Check out this tutorial for a walkthrough of how to do this and learn tricks such as:

  • Troubleshooting Kafka Connect JDBC sink
  • Un-nesting data for loading into the database (solves the error "(STRUCT) type doesn't have a mapping to the SQL database column type")
  • Handling timestamp fields
  • Idempotent database updates
  • Update rows in place based on a declared key

Whilst the tutorial is built around ksqlDB, the tips & tricks for the JDBC sink apply equally to Kafka Connect used on its own.

You can find the Docker Compose and notes on how to try this out for yourself here: https://rmoff.dev/ksqldb-jdbc-sink

Top comments (0)

🌚 Friends don't let friends browse without dark mode.

Just kidding, it's a personal preference. But you can change your theme, font, etc. in your settings.

The more you know. 🌈