DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost
hajirufai profile

I built a data-contract validator in pure Python (no pandas, no PyYAML) and it caught a 30% revenue ghost

#python #dataengineering #datascience #opensource
Comments
6 min read
A practical pipeline for turning messy business documents into spreadsheets
alex_zheng_49d3089c0d3fdf profile

A practical pipeline for turning messy business documents into spreadsheets

#dataengineering #ai #productivity
Comments
2 min read
Your Scraper Died at Row 12,000. The Rerun Pattern.
0012303 profile

Your Scraper Died at Row 12,000. The Rerun Pattern.

#python #webscraping #dataengineering #reliability
Comments
13 min read
Day 3 of 100 Days of ClickHouse® - ClickHouse® vs PostgreSQL
kanishga_subramani_49ad73 profile

Day 3 of 100 Days of ClickHouse® - ClickHouse® vs PostgreSQL

#clickhouse #dataengineering #devops #postgressql
2
Comments
2 min read
If the warehouse already has the data, why are we copying it elsewhere?
gazerro profile

If the warehouse already has the data, why are we copying it elsewhere?

#architecture #data #dataengineering #systemdesign
Comments
5 min read
What Is Agentic Workflow Consulting? A Practical Guide for Data Leaders
labyrinthanalytics profile

What Is Agentic Workflow Consulting? A Practical Guide for Data Leaders

#ai #machinelearning #dataengineering #llm
Comments
7 min read
Using Microsoft Fabric Shortcuts to Avoid Duplicate Data Copies
ravikiranpagidi profile

Using Microsoft Fabric Shortcuts to Avoid Duplicate Data Copies

#microsoftfabric #dataengineering #lakehouse #azure
Comments
7 min read
Part 2: Event Pipeline Design: The Real-Time Data Lifecycle from Kafka to Lakehouse
tayfun_yalcinkaya_9c29444 profile

Part 2: Event Pipeline Design: The Real-Time Data Lifecycle from Kafka to Lakehouse

#kafka #architecture #streaming #dataengineering
Comments
13 min read
I built a Databricks medallion lakehouse to roast my own YouTube history (Bronze Silver Gold Existential Dread)
shaanalpha profile

I built a Databricks medallion lakehouse to roast my own YouTube history (Bronze Silver Gold Existential Dread)

#showdev #dataengineering #python #databricks
1
Comments
3 min read
Bölüm 2: Event Pipeline Tasarımı: Kafka’dan Lakehouse’a Gerçek Zamanlı Veri Yaşam Döngüsü
tayfun_yalcinkaya_9c29444 profile

Bölüm 2: Event Pipeline Tasarımı: Kafka’dan Lakehouse’a Gerçek Zamanlı Veri Yaşam Döngüsü

#kafka #streaming #dataengineering #turkish
Comments
14 min read
Your Data Engineering Take-Home Is Free Labor
datadriven profile

Your Data Engineering Take-Home Is Free Labor

#dataengineering #interview #career #beginners
Comments
7 min read
Context Engineering: The Skill Replacing Prompt Engineering in 2026
gabrielhca profile

Context Engineering: The Skill Replacing Prompt Engineering in 2026

#ai #llm #python #dataengineering
Comments
4 min read
Why Dremio's Value Is Unique to Apache Iceberg Lakehouses and Agentic Analytics
alexmercedcoder profile

Why Dremio's Value Is Unique to Apache Iceberg Lakehouses and Agentic Analytics

#agents #ai #analytics #dataengineering
1
Comments
22 min read
The Phantom Schema Problem: Why Your Database Contract Breaks Before Your Tests Do
jitendra_devabhaktuni_0f1 profile

The Phantom Schema Problem: Why Your Database Contract Breaks Before Your Tests Do

#ai #database #datascience #dataengineering
Comments
4 min read
QN : Get started with lakehouses in Microsoft Fabric
pauletart profile

QN : Get started with lakehouses in Microsoft Fabric

#architecture #dataengineering #microsoft #tutorial
Comments
2 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.