loading...

FLaNK: Low Code Streaming: Populating Kafka Topics with FlinkSQL Joins in Real-Time

tspannhw profile image Timothy Spann Originally published at datainmotion.dev on ・2 min read


FLaNK: Low Code Streaming: Populating Kafka Topics with FlinkSQL Joins in Real-Time

FLaNK

Then I can create my 3 tables. Two are the source ones to join and the third is the destination for my insert.

INSERT INTO global_sensor_events

SELECT

scada.uuid,

scada.systemtime ,

scada.temperaturef ,

scada.pressure ,

scada.humidity ,

scada.lux ,

scada.proximity ,

scada.oxidising ,

scada.reducing ,

scada.nh3 ,

scada.gasko,

energy.current,

energy.voltage ,

energy.power ,

energy.total,

energy.fanstatus

FROM energy,

 scada

WHERE

scada.systemtime = energy.systemtime;

Examples

https://github.com/tspannhw/meetup-sensors/blob/master/flink-sql/

Assets / Scripts / DDL / SQL

https://github.com/tspannhw/FlinkSQLDemo

Flink Guide to SQL Joins
https://www.youtube.com/watch?v=5AuBlVRKQuo

Slides

https://www.slideshare.net/bunkertor/time-series-analysis-dataflow

Article on Joins

https://www.datainmotion.dev/2020/05/flink-sql-preview.html

Resources

Posted on by:

tspannhw profile

Timothy Spann

@tspannhw

I am a Principal Field Engineer for Data in Motion at Cloudera. I work with Apache NiFi, Apache Kafka, Apache Spark, Apache Flink, IoT, MXNet, DLJ.AI, Deep Learning, Machine Learning, Streaming...

Discussion

markdown guide