Skip to content

DEV Community

data-engineering Series' Articles

Back to Nithyalakshmi Kamalakkannan's Series

Cover image for End-to-End Real-Time Data Engineering on Databricks Using Spark Structured Streaming and Delta Lake

Nithyalakshmi Kamalakkannan

Jan 2

End-to-End Real-Time Data Engineering on Databricks Using Spark Structured Streaming and Delta Lake

#dataengineering #handson #realtimeproject #spark

1 min read

Cover image for Part 1: Creating Databricks Workspace and Enabling Unity Catalog

Nithyalakshmi Kamalakkannan

Jan 2

Part 1: Creating Databricks Workspace and Enabling Unity Catalog

#beginners #azure #dataengineering #tutorial

2 min read

Cover image for Part 2: Project Architecture

Nithyalakshmi Kamalakkannan

Jan 2

Part 2: Project Architecture

#systemdesign #data #architecture #dataengineering

2 min read

Cover image for Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

Nithyalakshmi Kamalakkannan

Jan 2

Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

#dataengineering #python #tutorial

1 min read

Cover image for Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

Nithyalakshmi Kamalakkannan

Jan 2

Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

#architecture #dataengineering #tutorial #sql

2 min read

Cover image for Part 5: Building a ZIP Code Dimension Table

Nithyalakshmi Kamalakkannan

Jan 2

Part 5: Building a ZIP Code Dimension Table

#analytics #database #dataengineering #tutorial

2 min read

Cover image for Part 6: Silver Layer – Cleansing, Enrichment, and Dimensions

Nithyalakshmi Kamalakkannan

Jan 2

Part 6: Silver Layer – Cleansing, Enrichment, and Dimensions

#analytics #architecture #dataengineering

2 min read

Cover image for Part 7: Gold Layer – Metrics, Watermarks, and Aggregations

Nithyalakshmi Kamalakkannan

Jan 2

Part 7: Gold Layer – Metrics, Watermarks, and Aggregations

#analytics #dataengineering #tutorial #python

2 min read

Cover image for Part 8: Databricks Pipeline & Dashboard

Nithyalakshmi Kamalakkannan

Jan 2

Part 8: Databricks Pipeline & Dashboard

#analytics #dataengineering #tutorial #sql

2 min read