DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
My take on transforming data
hyperwindmill profile
Danny

My take on transforming data

#javascript #opensource #webdev #dataengineering
Comments
3 min read
The Waterfall Pattern: A Tiered Strategy for Reliable Data Extraction
withatte profile

The Waterfall Pattern: A Tiered Strategy for Reliable Data Extraction

#webscraping #dataengineering #devops #dataextraction
1
Comments 1
5 min read
What Breaks When Kafka Meets Iceberg at Scale
novatechflow profile

What Breaks When Kafka Meets Iceberg at Scale

#kafka #dataengineering #iceberg #devops
Comments
4 min read
"Shannon Was Right, But We Can Be Smarter: How ALEC Achieves 22x Compression on IoT Data"
zeekmartin profile

"Shannon Was Right, But We Can Be Smarter: How ALEC Achieves 22x Compression on IoT Data"

#iot #rust #dataengineering #shannon
Comments
3 min read
Apache Data Lakehouse Weekly: January 15–22, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: January 15–22, 2026

#news #architecture #dataengineering #opensource
Comments
5 min read
Data Engineering Best Practices: The Complete Checklist
alexmercedcoder profile

Data Engineering Best Practices: The Complete Checklist

#architecture #data #dataengineering #productivity
4
Comments
4 min read
Batch vs. Streaming: Choose the Right Processing Model
alexmercedcoder profile

Batch vs. Streaming: Choose the Right Processing Model

#architecture #data #dataengineering #systemdesign
3
Comments
4 min read
Semantic Layer Best Practices: 7 Mistakes to Avoid
alexmercedcoder profile

Semantic Layer Best Practices: 7 Mistakes to Avoid

#analytics #architecture #data #dataengineering
1
Comments
5 min read
How to Think Like a Data Engineer
alexmercedcoder profile

How to Think Like a Data Engineer

#career #data #dataengineering #systemdesign
Comments
4 min read
Pipeline Observability: Know When Things Break
alexmercedcoder profile

Pipeline Observability: Know When Things Break

#data #dataengineering #devops #monitoring
2
Comments
4 min read
Partition and Organize Data for Performance
alexmercedcoder profile

Partition and Organize Data for Performance

#database #dataengineering #performance #sql
1
Comments
4 min read
Testing Data Pipelines: What to Validate and When
alexmercedcoder profile

Testing Data Pipelines: What to Validate and When

#data #dataengineering #softwareengineering #testing
1
Comments
4 min read
From AI Visibility to AI Governance: Building a Local-First LLM Cost & Risk Optimizer
harris_bashir_ac60ba5fff3 profile

From AI Visibility to AI Governance: Building a Local-First LLM Cost & Risk Optimizer

#ai #dataengineering #opensource #governance
Comments
4 min read
Snowflake's New Feature: AI-Powered Data Quality Checks You Can Set Up Directly in Snowsight
sagara profile

Snowflake's New Feature: AI-Powered Data Quality Checks You Can Set Up Directly in Snowsight

#snowflake #dataengineering
Comments
8 min read
Schema Evolution Without Breaking Consumers
alexmercedcoder profile

Schema Evolution Without Breaking Consumers

#api #architecture #database #dataengineering
1
Comments
4 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.