DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Apache Kafka — Deep Dive: Core Concepts, Data-Engineering Applications, and Real-World Production Practices

Apache Kafka — Deep Dive: Core Concepts, Data-Engineering Applications, and Real-World Production Practices

1
Comments
4 min read
Apache Iceberg Dev List Digest August 25-29

Apache Iceberg Dev List Digest August 25-29

Comments
5 min read
The Blueprint of a Data Team: Roles, Responsibilities, and Specializations

The Blueprint of a Data Team: Roles, Responsibilities, and Specializations

2
Comments
10 min read
Wait, what? Ingestion into silver?

Wait, what? Ingestion into silver?

Comments
1 min read
RIP Amazon Data Firehose Change Data Capture

RIP Amazon Data Firehose Change Data Capture

6
Comments 3
4 min read
Event-Driven Architectures on AWS: Beyond Lambda

Event-Driven Architectures on AWS: Beyond Lambda

4
Comments
2 min read
🔄 ETL vs ELT: What’s the Difference and Why It Matters?

🔄 ETL vs ELT: What’s the Difference and Why It Matters?

Comments
2 min read
Two Years of Microsoft Fabric: Game Changer or Still Leveling Up? 🚀

Two Years of Microsoft Fabric: Game Changer or Still Leveling Up? 🚀

1
Comments
2 min read
CDC in AWS: Content Data Capture from AWS RDS MySQL into AWS MSK Kafka topic using Debezium

CDC in AWS: Content Data Capture from AWS RDS MySQL into AWS MSK Kafka topic using Debezium

4
Comments
5 min read
LLPY-03: Extracción y Procesamiento Inteligente de Datos Legales

LLPY-03: Extracción y Procesamiento Inteligente de Datos Legales

Comments
21 min read
🏗️ The Role of a Data Engineer: Beyond Pipelines

🏗️ The Role of a Data Engineer: Beyond Pipelines

Comments
2 min read
Beyond Flat Tables: Model Hierarchical Data in Supabase with Recursive Queries

Beyond Flat Tables: Model Hierarchical Data in Supabase with Recursive Queries

1
Comments
7 min read
🌍 The Journey of Data: From Raw Logs to Insights

🌍 The Journey of Data: From Raw Logs to Insights

Comments
2 min read
🎯 The Challenge: Processing TBs of S3 Data Without Breaking the Bank

🎯 The Challenge: Processing TBs of S3 Data Without Breaking the Bank

Comments
5 min read
3 Things I Learned Building a Database from Scratch (And What I'd Do Differently)

3 Things I Learned Building a Database from Scratch (And What I'd Do Differently)

Comments
1 min read
Why Apache Iceberg is needed?

Why Apache Iceberg is needed?

1
Comments
6 min read
Do Caos à Orquestração: Como o DataOps Está Transformando Dados em Valor

Do Caos à Orquestração: Como o DataOps Está Transformando Dados em Valor

Comments
1 min read
International SaaS Nightmare: Timezone Edge Cases (And How to Solve Them Once and For All)

International SaaS Nightmare: Timezone Edge Cases (And How to Solve Them Once and For All)

Comments
2 min read
Paralelismo em Python para Engenharia de Dados: O Segredo das Tarefas I/O-Bound

Paralelismo em Python para Engenharia de Dados: O Segredo das Tarefas I/O-Bound

Comments 1
6 min read
Skip the Database: Building Analytics Dashboards Directly from S3 Files

Skip the Database: Building Analytics Dashboards Directly from S3 Files

15
Comments 3
9 min read
Apache Arrow dev list digest (Sept 8–12, 2025)

Apache Arrow dev list digest (Sept 8–12, 2025)

4
Comments
4 min read
Apache Iceberg dev list digest (Sept 8–12, 2025)

Apache Iceberg dev list digest (Sept 8–12, 2025)

1
Comments
5 min read
Anomaly Detection in Financial Transactions: Algorithms and Applications

Anomaly Detection in Financial Transactions: Algorithms and Applications

2
Comments
10 min read
The Ultimate Guide to Open Table Formats: Iceberg, Delta Lake, Hudi, Paimon, and DuckLake

The Ultimate Guide to Open Table Formats: Iceberg, Delta Lake, Hudi, Paimon, and DuckLake

1
Comments
16 min read
How to Pass the AWS Certified Data Engineer – Associate Exam2025

How to Pass the AWS Certified Data Engineer – Associate Exam2025

Comments
5 min read
loading...