DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Azure Data Factory — The Conveyor Belt of Data in the Cloud

Azure Data Factory — The Conveyor Belt of Data in the Cloud

Comments 1
5 min read
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Comments
4 min read
Data Engineering 101: Understanding Databases, Storage, and Security

Data Engineering 101: Understanding Databases, Storage, and Security

5
Comments
6 min read
10 Best Platforms to Learn Data Engineering in 2026

10 Best Platforms to Learn Data Engineering in 2026

Comments
4 min read
Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.

Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.

5
Comments 2
8 min read
Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

1
Comments
6 min read
Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

2
Comments
8 min read
Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows

Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows

Comments
9 min read
Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration

Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration

2
Comments
4 min read
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Comments
4 min read
Real-Time Crypto Data Pipeline

Real-Time Crypto Data Pipeline

2
Comments
3 min read
Another Data Nerd Guide to re:Invent 2025

Another Data Nerd Guide to re:Invent 2025

2
Comments
3 min read
Real-Time Cryptocurrency Data Pipeline

Real-Time Cryptocurrency Data Pipeline

Comments
5 min read
“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”

“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”

5
Comments
1 min read
From OLTP to OLAP: Streaming Databases into MotherDuck with Estuary

From OLTP to OLAP: Streaming Databases into MotherDuck with Estuary

1
Comments
7 min read
Crypto Real-Time Data Pipeline

Crypto Real-Time Data Pipeline

Comments
4 min read
Cryptocurrency Data Pipeline Project

Cryptocurrency Data Pipeline Project

Comments
4 min read
Apache Kafka & Amazon MSK: The Beating Heart of Real-Time Data

Apache Kafka & Amazon MSK: The Beating Heart of Real-Time Data

1
Comments
4 min read
Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Comments
3 min read
Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Comments
3 min read
Real-Time Crypto Data Pipeline

Real-Time Crypto Data Pipeline

Comments
3 min read
Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

Comments
2 min read
Streaming Data Using Apache Kafka

Streaming Data Using Apache Kafka

1
Comments
2 min read
SUPCON Uses SeaTunnel to Build an Efficient Data Collection Framework, Achieving 0 Failures in Core Data Synchronization Tasks!

SUPCON Uses SeaTunnel to Build an Efficient Data Collection Framework, Achieving 0 Failures in Core Data Synchronization Tasks!

Comments
14 min read
Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search

Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search

5
Comments
7 min read
loading...