DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
🚀 Synthetic Data: The Next Frontier for Data Engineers

🚀 Synthetic Data: The Next Frontier for Data Engineers

Comments
2 min read
Pytest: Como Testar Módulos Python com Configuração no Nível Superior

Pytest: Como Testar Módulos Python com Configuração no Nível Superior

Comments
5 min read
Databend Monthly Report: July 2025

Databend Monthly Report: July 2025

Comments
3 min read
Building AI-Powered Data Pipelines: Where Data Engineering Meets Machine Learning

Building AI-Powered Data Pipelines: Where Data Engineering Meets Machine Learning

Comments
2 min read
Where We Encounter Delimited Data and How We Handle It

Where We Encounter Delimited Data and How We Handle It

4
Comments
6 min read
wget vs. curl: when to use which?

wget vs. curl: when to use which?

Comments
2 min read
🔐 Data Governance: From Chaos to Control

🔐 Data Governance: From Chaos to Control

Comments
2 min read
Building a Data Mart in Amazon Redshift: A Practical Guide

Building a Data Mart in Amazon Redshift: A Practical Guide

Comments
6 min read
Apache Arrow dev list digest (Aug 25–29 2025)

Apache Arrow dev list digest (Aug 25–29 2025)

Comments
4 min read
Revamping Real-Time Data Ingestion for Scalable Media Intelligence

Revamping Real-Time Data Ingestion for Scalable Media Intelligence

3
Comments
4 min read
Scraping the Schema of NetSuite

Scraping the Schema of NetSuite

Comments
2 min read
You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

2
Comments 1
8 min read
Docker for Data Engineers: The Complete Beginner’s Guide

Docker for Data Engineers: The Complete Beginner’s Guide

5
Comments
6 min read
Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

3
Comments
5 min read
Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

3
Comments 1
5 min read
Career Opportunities After Completing AI & Data Science Degree

Career Opportunities After Completing AI & Data Science Degree

Comments
3 min read
Big Data Fundamentals: real-time analytics project

Big Data Fundamentals: real-time analytics project

Comments
6 min read
The Case for Apache Airflow and Kafka in Data Engineering

The Case for Apache Airflow and Kafka in Data Engineering

1
Comments
2 min read
🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

Comments
2 min read
Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

4
Comments
4 min read
Real-Time Fraud Detection Using Kafka and Machine Learning

Real-Time Fraud Detection Using Kafka and Machine Learning

Comments
5 min read
🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

Comments
3 min read
Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

1
Comments
3 min read
15 Core Concepts of Data Engineering

15 Core Concepts of Data Engineering

Comments
9 min read
Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Comments
3 min read
loading...