DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Building a Data Mart in Amazon Redshift: A Practical Guide

Building a Data Mart in Amazon Redshift: A Practical Guide

Comments
6 min read
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

1
Comments 1
6 min read
Apache Arrow dev list digest (Aug 25–29 2025)

Apache Arrow dev list digest (Aug 25–29 2025)

Comments
4 min read
Revamping Real-Time Data Ingestion for Scalable Media Intelligence

Revamping Real-Time Data Ingestion for Scalable Media Intelligence

Comments
4 min read
Scraping the Schema of NetSuite

Scraping the Schema of NetSuite

Comments
2 min read
You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

2
Comments 1
8 min read
Docker for Data Engineers: The Complete Beginner’s Guide

Docker for Data Engineers: The Complete Beginner’s Guide

5
Comments
6 min read
Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

3
Comments
5 min read
Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

3
Comments 1
5 min read
Career Opportunities After Completing AI & Data Science Degree

Career Opportunities After Completing AI & Data Science Degree

Comments
3 min read
Building an End-to-End Data Engineering Pipeline with DuckDB and Python

Building an End-to-End Data Engineering Pipeline with DuckDB and Python

1
Comments
5 min read
Big Data Fundamentals: real-time analytics project

Big Data Fundamentals: real-time analytics project

Comments
6 min read
The Case for Apache Airflow and Kafka in Data Engineering

The Case for Apache Airflow and Kafka in Data Engineering

1
Comments
2 min read
🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

Comments
2 min read
Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

4
Comments
4 min read
What is the Modern Data Stack?

What is the Modern Data Stack?

Comments
3 min read
Real-Time Fraud Detection Using Kafka and Machine Learning

Real-Time Fraud Detection Using Kafka and Machine Learning

Comments
5 min read
🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

Comments
3 min read
Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

1
Comments
3 min read
15 Core Concepts of Data Engineering

15 Core Concepts of Data Engineering

Comments
9 min read
Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Comments
3 min read
Understanding Data Warehousing for Retail Analytics: A Comprehensive Guide

Understanding Data Warehousing for Retail Analytics: A Comprehensive Guide

1
Comments
3 min read
Building a Modern Data Warehouse in SQL Server with Medallion Architecture

Building a Modern Data Warehouse in SQL Server with Medallion Architecture

Comments
11 min read
Benefits of OLAP and OLTP in Data Management.

Benefits of OLAP and OLTP in Data Management.

Comments
2 min read
Building High-Load API Services in Go: From Design to Production

Building High-Load API Services in Go: From Design to Production

2
Comments
23 min read
loading...