Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Kunwar Jhamat

Mar 5

Iterator Patterns: How to Process Millions of Records Without Running Out of Memory

#php #programming #dataengineering #performance

5 min read

Mohamed Hussain S

Apr 7

Why My Metrics Pipeline with Telegraf Didn’t Work (and What I Learned)

#dataengineering #clickhouse #devops #observability

2 min read

BUKYA NARESH

Apr 7

Python was too slow for 10M rows—So I built a C-Bridge (and found the hidden data loss)

#python #cpp #performance #dataengineering

2 min read

Vibhu Gupta

Mar 8

9 Data Engineering Challenges That Kill Pipelines in Production (And How I approached Them With Pure Snowflake SQL)

#snowflake #dataengineering #sql #datapipeline

14 min read

Vignesh

Apr 7

How I stopped bad data from reaching my warehouse using a single Airflow task

#dataengineering #airflow #python #api

4 min read

Rocio Baigorria

Apr 6

Stop Babysitting Servers: Build a Scalable Serverless Data Lake on AWS

#showdev #aws #serverless #dataengineering

2 min read

shubham pandey (Connoisseur)

Mar 8

History of Kafka the message broker

#architecture #dataengineering #distributedsystems #systemdesign

3 min read

Joni Sar

Apr 6

Introducing QueryFlux: Open-Source Universal Multi-Engine Query Router and SQL Proxy

#database #dataengineering #devops #opensource

7 min read

Lenin Mishra

Apr 6

How I Built a Performance Dashboard for a Multi-Office Chiropractic Practice

#googlesheets #analytics #looker #dataengineering

5 min read

Philip McClarence

Mar 3

Optimizing Continuous Aggregate Performance for Large Datasets

#database #dataengineering #performance #postgres

4 min read

Tejaswita Soni

Mar 3

How to Build a Real-Time DynamoDB to S3 Analytics Pipeline with Apache Iceberg

#analytics #aws #dataengineering #tutorial

8 min read

David Marcelo Petrocelli

Mar 3

How Netflix Turns 2 Trillion Daily Events Into Architectural Decisions (And How You Can Too)

#architecture #microservices #dataengineering #netflix

14 min read

Daniel for Datalaria

Mar 7

S&OP Engineering II: Demand Planning from Guessing to Probability

#analytics #dataengineering #datascience #machinelearning

4 min read

Brandon Lozano

Mar 3

Debugging & Refactoring Spaghetti Code - The A.I Special 🤖

#ai #programming #dataengineering #api

5 min read

Damaa-C

Mar 6

Mastering SQL Joins and Window Functions: A Practical Guide with Example Data

#learning #dataengineering #data #developer

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# dataengineering

Iterator Patterns: How to Process Millions of Records Without Running Out of Memory

Why My Metrics Pipeline with Telegraf Didn’t Work (and What I Learned)

Python was too slow for 10M rows—So I built a C-Bridge (and found the hidden data loss)

9 Data Engineering Challenges That Kill Pipelines in Production (And How I approached Them With Pure Snowflake SQL)

How I stopped bad data from reaching my warehouse using a single Airflow task

Stop Babysitting Servers: Build a Scalable Serverless Data Lake on AWS

History of Kafka the message broker

Introducing QueryFlux: Open-Source Universal Multi-Engine Query Router and SQL Proxy

How I Built a Performance Dashboard for a Multi-Office Chiropractic Practice

Optimizing Continuous Aggregate Performance for Large Datasets

How to Build a Real-Time DynamoDB to S3 Analytics Pipeline with Apache Iceberg

How Netflix Turns 2 Trillion Daily Events Into Architectural Decisions (And How You Can Too)

S&OP Engineering II: Demand Planning from Guessing to Probability

Debugging & Refactoring Spaghetti Code - The A.I Special 🤖

Mastering SQL Joins and Window Functions: A Practical Guide with Example Data