DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Building a Data Platform on AWS: Essential Design Considerations for Power BI

Building a Data Platform on AWS: Essential Design Considerations for Power BI

3
Comments
5 min read
Build a Complete Data Pipeline from Scratch: CSV to Dashboard Using Python, MySQL, and Airflow”

Build a Complete Data Pipeline from Scratch: CSV to Dashboard Using Python, MySQL, and Airflow”

2
Comments 1
3 min read
The Future of Data Pipelines: How AI Is Redefining ETL Forever

The Future of Data Pipelines: How AI Is Redefining ETL Forever

1
Comments
4 min read
Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™

Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™

1
Comments
10 min read
Building Custom MCP Servers with Python: A Data Engineer's Guide 🛠️

Building Custom MCP Servers with Python: A Data Engineer's Guide 🛠️

Comments 1
5 min read
The Next Era of Databases: When Queries Write Themselves

The Next Era of Databases: When Queries Write Themselves

Comments
4 min read
Why 71,000 Data Engineers Read My Article: What I Learned About Technical Writing

Why 71,000 Data Engineers Read My Article: What I Learned About Technical Writing

4
Comments 1
6 min read
Transforming Tableau Performance: How Optimized Data Logic Cut Dashboard Load Time by 98.9%

Transforming Tableau Performance: How Optimized Data Logic Cut Dashboard Load Time by 98.9%

5
Comments
8 min read
rec

rec

Comments
2 min read
How We Cut LLM Batch Inference Time in Half with Dynamic Prefix Bucketing

How We Cut LLM Batch Inference Time in Half with Dynamic Prefix Bucketing

5
Comments
13 min read
Let's say you have a data lake

Let's say you have a data lake

Comments
3 min read
Azure Data Factory for ETL

Azure Data Factory for ETL

Comments
5 min read
FlightPath Server Has Landed

FlightPath Server Has Landed

Comments
1 min read
1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL

1 billion JSON records, 1-second query response: Apache Doris vs. ClickHouse, Elasticsearch, and PostgreSQL

5
Comments
7 min read
How to Avoid Common Data Management Pitfalls in Enterprise RAG Systems: A Guide to Effective Governance and Observability

How to Avoid Common Data Management Pitfalls in Enterprise RAG Systems: A Guide to Effective Governance and Observability

Comments
5 min read
🔥 Day 6: Essential PySpark DataFrame Transformations

🔥 Day 6: Essential PySpark DataFrame Transformations

Comments
2 min read
Building Self-Healing, Reliable Data Pipelines That Think

Building Self-Healing, Reliable Data Pipelines That Think

Comments
4 min read
How to Data Engineer the ETLFunnel Way

How to Data Engineer the ETLFunnel Way

Comments
5 min read
How to Data Engineer the ETLFunnel Way

How to Data Engineer the ETLFunnel Way

Comments
3 min read
How to Data Engineer the ETLFunnel Way

How to Data Engineer the ETLFunnel Way

Comments
5 min read
A Dashboard About Scammers, Telemarketers, My Cellphone, and Who Annoys Me Most

A Dashboard About Scammers, Telemarketers, My Cellphone, and Who Annoys Me Most

Comments
3 min read
How to Build Reliable Data Pipelines for Analytics

How to Build Reliable Data Pipelines for Analytics

Comments
1 min read
From Postgres to Iceberg

From Postgres to Iceberg

1
Comments
11 min read
isql

isql

Comments
1 min read
The data lakehouse evolution

The data lakehouse evolution

Comments
11 min read
loading...