Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

John Wakaba

Nov 4 '25

🌍 Automating Africa’s Energy Data Collection Using Python, Playwright(+Why Playwright ?), and MongoDB (2000–2024)

#dataengineering #mongodb #python #automation

5 min read

Byron Hsieh

Nov 6 '25

From 8 Minutes to 40 Seconds: Solving Data Pipeline Deployment Bottlenecks with Git Sparse Checkout

#git #devops #dataengineering #azure

5 min read

lotanna obianefo

Oct 2 '25

Create a Microsoft Fabric Lakehouse

#database #dataengineering #cloudcomputing #datascience

6 min read

Farhan Khan

Oct 2 '25

Core Concepts of Kafka

#architecture #beginners #dataengineering

8 min read

elisha lukalia

Oct 2 '25

From Kafka to Clean Tables: Building a Confluent Snowflake Pipeline with Streams & Tasks.

#dataengineering #automation #tutorial #sql

9 min read

Leanid Herasimau

Oct 3 '25

Apache Kafka: ZooKeeper vs. KRaft — A Complete Comparison of Approaches

#architecture #backend #dataengineering

6 min read

John Kioko

Oct 6 '25

Introduction to Apache Airflow

#dataengineering #beginners #learning #python

4 min read

André Paris

Oct 14 '25

Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK

#aws #dataengineering #typescript #postgres

8 min read

Brian Misachi

Nov 5 '25

From Postgres to Iceberg

#database #postgres #dataengineering

11 min read

Lagat Josiah

Nov 4 '25

Real-Time Cryptocurrency Data Pipeline

#cryptocurrency #dataengineering #python #monitoring

12 min read

Kuldeep Paul

Oct 14 '25

Synthetic Data for RAG: Safe Generation, Deduplication, and Drift-Aware Curation in 2025

#dataengineering #rag #testing #llm

10 min read

Sagara

Oct 1 '25

Personal Picks: Data Product News (October 1, 2025)

#dataengineering #snowflake #databricks

7 min read

Apache Doris

Nov 4 '25

1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL

#bigdata #database #olap #dataengineering

7 min read

David Kershaw

Nov 4 '25

SQL: is there a better way to code this?

#sql #csv #database #dataengineering

1 min read

Lakshmi Narayana Rasalay

Oct 4 '25

Building Real-Time Data Pipelines from PostgreSQL Using Flink CDC

#architecture #dataengineering #postgres

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# dataengineering

🌍 Automating Africa’s Energy Data Collection Using Python, Playwright(+Why Playwright ?), and MongoDB (2000–2024)

From 8 Minutes to 40 Seconds: Solving Data Pipeline Deployment Bottlenecks with Git Sparse Checkout

Create a Microsoft Fabric Lakehouse

Core Concepts of Kafka

From Kafka to Clean Tables: Building a Confluent Snowflake Pipeline with Streams & Tasks.

Apache Kafka: ZooKeeper vs. KRaft — A Complete Comparison of Approaches

Introduction to Apache Airflow

Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK

From Postgres to Iceberg

Real-Time Cryptocurrency Data Pipeline

Synthetic Data for RAG: Safe Generation, Deduplication, and Drift-Aware Curation in 2025

Personal Picks: Data Product News (October 1, 2025)

1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL

SQL: is there a better way to code this?

Building Real-Time Data Pipelines from PostgreSQL Using Flink CDC