DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Introduction to Apache Airflow
mutindakioko profile

Introduction to Apache Airflow

#dataengineering #beginners #learning #python
1
Comments
4 min read
Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK
andreparis profile

Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK

#aws #dataengineering #typescript #postgres
2
Comments
8 min read
From Postgres to Iceberg
misachi profile

From Postgres to Iceberg

#database #postgres #dataengineering
1
Comments
11 min read
Real-Time Cryptocurrency Data Pipeline
lagat_josiah_f024a2c855bc profile

Real-Time Cryptocurrency Data Pipeline

#cryptocurrency #dataengineering #python #monitoring
Comments
12 min read
Synthetic Data for RAG: Safe Generation, Deduplication, and Drift-Aware Curation in 2025
kuldeep_paul profile

Synthetic Data for RAG: Safe Generation, Deduplication, and Drift-Aware Curation in 2025

#dataengineering #rag #testing #llm
2
Comments
10 min read
Personal Picks: Data Product News (October 1, 2025)
sagara profile

Personal Picks: Data Product News (October 1, 2025)

#dataengineering #snowflake #databricks
Comments
7 min read
1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL
apachedoris profile

1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL

#bigdata #database #olap #dataengineering
6
Comments
7 min read
SQL: is there a better way to code this?
david_kershaw_b6916404da6 profile

SQL: is there a better way to code this?

#sql #csv #database #dataengineering
Comments 1
1 min read
Building Real-Time Data Pipelines from PostgreSQL Using Flink CDC
lakshminarayan_r_6f07f9c0 profile

Building Real-Time Data Pipelines from PostgreSQL Using Flink CDC

#architecture #dataengineering #postgres
Comments
5 min read
How to Convert Excel to CSV in Python using Spire.XLS for Python
jelizaveta profile

How to Convert Excel to CSV in Python using Spire.XLS for Python

#dataengineering #automation #python #tutorial
Comments
4 min read
Building a Sales Database in PostgreSQL — Schema, Data & JOIN Examples
sajjadrahman56 profile

Building a Sales Database in PostgreSQL — Schema, Data & JOIN Examples

#database #postgres #dataengineering #sql
4
Comments
6 min read
Building Self-Healing, Reliable Data Pipelines That Think
shailaputri profile

Building Self-Healing, Reliable Data Pipelines That Think

#automation #agents #ai #dataengineering
Comments 1
4 min read
Interesting links - September 2025
rmoff profile

Interesting links - September 2025

#data #dataengineering #kafka #flink
Comments
13 min read
Beyond the Browser: Crafting a Robust Web Scraping Pipeline for Dynamic Sports Data
ffteamnames profile

Beyond the Browser: Crafting a Robust Web Scraping Pipeline for Dynamic Sports Data

#architecture #automation #dataengineering
Comments 1
3 min read
Get Started with Fastest SQL Query Engine - Presto C++ (Prestissimo): Beginner Friendly Setup Guide with Docker.
saurabhmahawar profile

Get Started with Fastest SQL Query Engine - Presto C++ (Prestissimo): Beginner Friendly Setup Guide with Docker.

#opensource #queryengine #presto #dataengineering
Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.