DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
SIMPLE BEGINNER CRYPTO ETL PIPELINE.
wangeci_ndovu profile

SIMPLE BEGINNER CRYPTO ETL PIPELINE.

#beginners #dataengineering #postgres #python
1
Comments
2 min read
Why My Spark Container Keeps Exiting — Docker PID 1 and the Daemon Trap
lee_yao_cfeb14fb9b141b8c5 profile

Why My Spark Container Keeps Exiting — Docker PID 1 and the Daemon Trap

#docker #spark #dataengineering #devops
Comments 1
5 min read
The Backyard Quarry, Part 8: From Rocks to Reality
kenwalger profile

The Backyard Quarry, Part 8: From Rocks to Reality

#architecture #dataengineering #sideprojects #systemdesign
2
Comments
4 min read
Using Databricks Genie for Natural Language Querying on Semantic Models
ravikiranpagidi profile

Using Databricks Genie for Natural Language Querying on Semantic Models

#databricks #machinelearning #dataengineering #analytics
Comments
7 min read
The Web Scraping Data Warehouse Pipeline I Run for $12/Month
vhub_systems_ed5641f65d59 profile

The Web Scraping Data Warehouse Pipeline I Run for $12/Month

#dataengineering #python #tutorial #webdev
Comments
3 min read
The real problem with ingesting MongoDB into Delta Lake (and how I built a library to fix it)
lholiv profile

The real problem with ingesting MongoDB into Delta Lake (and how I built a library to fix it)

#dataengineering #python #opensource #mongodb
2
Comments 7
5 min read
How to Think During a Data Engineering Interview in 2026
datadriven profile

How to Think During a Data Engineering Interview in 2026

#dataengineering #interview #career #python
1
Comments
6 min read
I Analyzed 10 Million Records in 47 Seconds Using Python + DuckDB (No Spark, No Cloud)
dattasable profile

I Analyzed 10 Million Records in 47 Seconds Using Python + DuckDB (No Spark, No Cloud)

#sql #python #dataengineering #tutorial
3
Comments 2
3 min read
CSV: The Format Nobody Designed
vivian-voss profile

CSV: The Format Nobody Designed

#csv #dataengineering #unix #architecture
Comments
2 min read
Apache Data Lakehouse Weekly: April 30–May 6, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: April 30–May 6, 2026

#news #data #dataengineering #opensource
Comments
9 min read
Performance and Apache Iceberg's Metadata
alexmercedcoder profile

Performance and Apache Iceberg's Metadata

#architecture #database #dataengineering #performance
Comments
7 min read
We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed
tasrieitservices profile

We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed

#airflow #dataengineering #kubernetes #eks
1
Comments
12 min read
Stop Using Subqueries: 3 Advanced SQL CTE Patterns That Saved My Production Database
dattasable profile

Stop Using Subqueries: 3 Advanced SQL CTE Patterns That Saved My Production Database

#sql #database #dataengineering #programming
Comments 1
3 min read
CTEs vs Subqueries in SQL: Which One Should You Use?
neha_christina_1ac8651819 profile

CTEs vs Subqueries in SQL: Which One Should You Use?

#sql #dataengineering #beginners #programming
Comments
3 min read
Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job
luca1iu profile
Jin

Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job

#azure #dataanalyst #functions #dataengineering
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.