DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
A Real-Time Earthquake Monitoring Pipeline with Kafka, MySQL, PostgreSQL, and Grafana

A Real-Time Earthquake Monitoring Pipeline with Kafka, MySQL, PostgreSQL, and Grafana

3
Comments
4 min read
Pandas vs Polars: Is It Time to Rethink Python’s Trusted DataFrame Library?

Pandas vs Polars: Is It Time to Rethink Python’s Trusted DataFrame Library?

3
Comments 2
3 min read
Big Data Fundamentals: big data

Big Data Fundamentals: big data

5
Comments
6 min read
Big Data Fundamentals: big data example

Big Data Fundamentals: big data example

5
Comments
5 min read
Big Data Fundamentals: big data project

Big Data Fundamentals: big data project

5
Comments
5 min read
Big Data Fundamentals: big data tutorial

Big Data Fundamentals: big data tutorial

5
Comments
5 min read
Why Your Data Fails You - and How a Data Platform Can Fix It

Why Your Data Fails You - and How a Data Platform Can Fix It

1
Comments
4 min read
Cloud Data Tools Simplified: AWS, Google Cloud, and Azure

Cloud Data Tools Simplified: AWS, Google Cloud, and Azure

1
Comments
7 min read
Understanding Consistency in PostgreSQL: A Deep Dive into the “C” in ACID

Understanding Consistency in PostgreSQL: A Deep Dive into the “C” in ACID

Comments
3 min read
Building a Real-Time Crypto Pipeline with Binance APIs, PostgreSQL, Debezium, Kafka, Spark & Cassandra

Building a Real-Time Crypto Pipeline with Binance APIs, PostgreSQL, Debezium, Kafka, Spark & Cassandra

2
Comments
6 min read
BEGINNER'S GUIDE TO STREAM REAL-TIME DATA USING APACHE KAFKA

BEGINNER'S GUIDE TO STREAM REAL-TIME DATA USING APACHE KAFKA

Comments
4 min read
Stop Drawing ETL Diagrams — Your Python Code Visualizes Itself

Stop Drawing ETL Diagrams — Your Python Code Visualizes Itself

4
Comments
4 min read
⚡ Kafka ClickHouse: Real-Time Data Pipeline for Beginners

⚡ Kafka ClickHouse: Real-Time Data Pipeline for Beginners

2
Comments
2 min read
Become the Serverless DJ. How to process audio using AWS?

Become the Serverless DJ. How to process audio using AWS?

2
Comments
8 min read
Discussion about Data Science project idea

Discussion about Data Science project idea

Comments
1 min read
Why Data Formats Matter More Than You Think

Why Data Formats Matter More Than You Think

1
Comments
19 min read
Troubleshooting SeaTunnel Cluster Split-Brain: A Deep Dive into Hazelcast Configuration and GC-Induced Failures

Troubleshooting SeaTunnel Cluster Split-Brain: A Deep Dive into Hazelcast Configuration and GC-Induced Failures

Comments
9 min read
A Simple Fix for SeaTunnel Excel Failing to Convert Numeric Types to Strings | With Source Code Packaging

A Simple Fix for SeaTunnel Excel Failing to Convert Numeric Types to Strings | With Source Code Packaging

Comments
3 min read
Personal Picks: Data Product News (May 28, 2025)

Personal Picks: Data Product News (May 28, 2025)

Comments
7 min read
Big Data Fundamentals: spark example

Big Data Fundamentals: spark example

1
Comments
5 min read
Building Multi-Tenant Analytics with Snowflake RBAC and Sigma Computing: Part 3

Building Multi-Tenant Analytics with Snowflake RBAC and Sigma Computing: Part 3

Comments
3 min read
Big Data: Distributed Computing - Your Essential Resource Guide

Big Data: Distributed Computing - Your Essential Resource Guide

Comments
3 min read
🚀 Dagster 2025: Not Just ETL — A Data Asset Mindset

🚀 Dagster 2025: Not Just ETL — A Data Asset Mindset

Comments
1 min read
Big Data Fundamentals: hadoop tutorial

Big Data Fundamentals: hadoop tutorial

2
Comments
6 min read
Top 5 Open Source Tools Every Data Engineer Should Know About (2025 Edition)

Top 5 Open Source Tools Every Data Engineer Should Know About (2025 Edition)

Comments
3 min read
loading...