DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Designing Data-Intensive Applications — Chapter 1: Reliable, Scalable, and Maintainable Applications

Designing Data-Intensive Applications — Chapter 1: Reliable, Scalable, and Maintainable Applications

Comments
4 min read
End-to-End Data Workflow: Kestra, Redshift, and dbt Integration

End-to-End Data Workflow: Kestra, Redshift, and dbt Integration

1
Comments
9 min read
Fixing Type Hints for Callable Objects with Custom Signatures in Dagster

Fixing Type Hints for Callable Objects with Custom Signatures in Dagster

1
Comments
3 min read
From Dashboards to Decisions: Building Scalable Self-Service BI for Real Impact

From Dashboards to Decisions: Building Scalable Self-Service BI for Real Impact

Comments
2 min read
Real-World Strategies for Scaling AI in Large Organizations

Real-World Strategies for Scaling AI in Large Organizations

Comments
3 min read
Introducing ReelTrust: What if data engineering could solve our AI deepfakes problem?

Introducing ReelTrust: What if data engineering could solve our AI deepfakes problem?

Comments
5 min read
Who Needs Real-Time Streaming? Use Cases & Architecture Across Industries

Who Needs Real-Time Streaming? Use Cases & Architecture Across Industries

Comments
8 min read
Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

Comments
6 min read
From “I want automation” to “It runs”: 15 decisions for lead enrichment that actually execute

From “I want automation” to “It runs”: 15 decisions for lead enrichment that actually execute

Comments
3 min read
Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing

Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing

10
Comments
6 min read
Real-Time Cryptocurrency Data Pipeline

Real-Time Cryptocurrency Data Pipeline

Comments
5 min read
What Does a Data Engineer Do? Understanding the Role and Career Path

What Does a Data Engineer Do? Understanding the Role and Career Path

1
Comments
6 min read
Decommissioning the Dinosaur: A 4-Phase Playbook for Migrating Your Legacy Data Warehouse to Databricks

Decommissioning the Dinosaur: A 4-Phase Playbook for Migrating Your Legacy Data Warehouse to Databricks

Comments
4 min read
Crypto Real-Time Data Pipeline

Crypto Real-Time Data Pipeline

Comments
4 min read
Cryptocurrency Data Pipeline Project

Cryptocurrency Data Pipeline Project

Comments
4 min read
Set up an open-source AI analyst for PostgreSQL in 2 minutes

Set up an open-source AI analyst for PostgreSQL in 2 minutes

1
Comments
5 min read
The Semantic Gap in Data Quality: Why Your Monitoring is Lying to You

The Semantic Gap in Data Quality: Why Your Monitoring is Lying to You

1
Comments 1
7 min read
A Deep Dive into Apache Spark Architecture

A Deep Dive into Apache Spark Architecture

1
Comments
4 min read
Building an Automated Data Pipeline: Injuries vs Performance in the Premier League

Building an Automated Data Pipeline: Injuries vs Performance in the Premier League

Comments
6 min read
2025-2026 Guide to Learning about Apache Iceberg, Data Lakehouse & Agentic AI

2025-2026 Guide to Learning about Apache Iceberg, Data Lakehouse & Agentic AI

Comments
9 min read
Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

1
Comments
8 min read
Evolution of Processing: SPL One-Click Acceleration for Log-to-Metric Conversion

Evolution of Processing: SPL One-Click Acceleration for Log-to-Metric Conversion

Comments
6 min read
Hands-On ACID in PostgreSQL : Part-1

Hands-On ACID in PostgreSQL : Part-1

Comments
3 min read
My First Data Engineering Project: Building a Real-Time IoT Pipeline on Azure

My First Data Engineering Project: Building a Real-Time IoT Pipeline on Azure

Comments
6 min read
Data Engineering 102: Understanding Transactions, ACID, and Isolation in PostgreSQL

Data Engineering 102: Understanding Transactions, ACID, and Isolation in PostgreSQL

2
Comments
5 min read
loading...