DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Running Apache Airflow + Docker for Free Using GitHub Codespaces
tanmay_bhurkunde profile
Tanmay

Running Apache Airflow + Docker for Free Using GitHub Codespaces

#dataengineering #docker #python #apacheairflow
Comments
1 min read
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
alexmercedcoder profile

Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take

#architecture #data #database #dataengineering
Comments 1
21 min read
Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast
alexmercedcoder profile

Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast

#architecture #dataengineering #infrastructure #performance
2
Comments 1
22 min read
The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted
alexmercedcoder profile

The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted

#ai #analytics #data #dataengineering
Comments
22 min read
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
gathurum profile

ETL Pipeline: Fetching Real-Time News Data with Python and Postgres

#dataengineering #api #etl #beginners
1
Comments
4 min read
I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast
hajirufai profile

I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast

#dataengineering #python #database #performance
Comments
6 min read
Why I Don’t Let the LLM Decide Issue State
ahmadhumayun profile

Why I Don’t Let the LLM Decide Issue State

#ai #python #dataengineering #marketinganalytics
Comments
5 min read
The customer_id that isn't a customer
gaur_data_c6511af396c4bce profile

The customer_id that isn't a customer

#agents #analytics #dataengineering #sql
Comments
8 min read
Your Scraper Collected 50 Rows. There Were 4,000.
0012303 profile

Your Scraper Collected 50 Rows. There Were 4,000.

#webscraping #python #dataengineering #pagination
Comments
7 min read
Deeper into Dataform 3: Auditing Dataform
ben_watson_data profile

Deeper into Dataform 3: Auditing Dataform

#dataform #dataengineering #gcp #bigquery
Comments
2 min read
How I Broke Down My ETL Pipeline Project Into Smaller Engineering Exercises
tanmay_bhurkunde profile
Tanmay

How I Broke Down My ETL Pipeline Project Into Smaller Engineering Exercises

#dataengineering #database #etl #sql
Comments
2 min read
Apache Data Lakehouse Weekly: July 1 to July 8, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: July 1 to July 8, 2026

#news #data #dataengineering #opensource
2
Comments 1
24 min read
I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost
hajirufai profile

I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost

#python #dataengineering #datascience #opensource
Comments
6 min read
A practical pipeline for turning messy business documents into spreadsheets
alex_zheng_49d3089c0d3fdf profile

A practical pipeline for turning messy business documents into spreadsheets

#dataengineering #ai #productivity
Comments
2 min read
A Fair Coin Isn't Enough: When a Perfectly Randomized Experiment Is Impossible to Analyze
hexisteme profile
John

A Fair Coin Isn't Enough: When a Perfectly Randomized Experiment Is Impossible to Analyze

#analytics #data #dataengineering #datascience
1
Comments 2
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.