DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Google's LEGO tribute 🧩
jigyasagrover profile

Google's LEGO tribute 🧩

#computerscience #dataengineering #google #systemdesign
27
Comments 8
1 min read
The Ultimate Linux Command Cheat Sheet for Data Engineers and Analysts
grayhat profile

The Ultimate Linux Command Cheat Sheet for Data Engineers and Analysts

#dataengineering #bash #datascience #python
85
Comments 4
4 min read
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)
olgabraginskaya profile

When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)

#dataengineering #python #data #tutorial
17
Comments 2
5 min read
Join Data from Anywhere: The Streaming SQL Engine That Bridges Databases, APIs, and Files
theodore_p_9749548f7dd03 profile

Join Data from Anywhere: The Streaming SQL Engine That Bridges Databases, APIs, and Files

#python #database #dataengineering #sql
8
Comments 1
17 min read
Why 71,000 Data Engineers Read My Article: What I Learned About Technical Writing
pradeep_kaalluri profile

Why 71,000 Data Engineers Read My Article: What I Learned About Technical Writing

#dataengineering #writing #programming #career
4
Comments 1
6 min read
Real-Time is an SLA, Not an Architecture: When You Actually Need Kafka (And When You Don't)
qvfagundes profile

Real-Time is an SLA, Not an Architecture: When You Actually Need Kafka (And When You Don't)

#discuss #architecture #dataengineering #career
1
Comments
10 min read
🌍 Automating Africa’s Energy Data Collection Using Python, Playwright(+Why Playwright ?), and MongoDB (2000–2024)
john_analytics profile

🌍 Automating Africa’s Energy Data Collection Using Python, Playwright(+Why Playwright ?), and MongoDB (2000–2024)

#dataengineering #mongodb #python #automation
5
Comments
5 min read
S3-Native Kafka Alternatives: What's Actually Different
novatechflow profile

S3-Native Kafka Alternatives: What's Actually Different

#kafka #dataengineering #opensource #devops
Comments
3 min read
Why Parquet Is Everywhere - And What Makes It Actually Fast?
mohhddhassan profile

Why Parquet Is Everywhere - And What Makes It Actually Fast?

#dataengineering #parquet #bigdata #dataarchitecture
2
Comments
3 min read
RIP Amazon Data Firehose Change Data Capture
psantus profile

RIP Amazon Data Firehose Change Data Capture

#dataengineering #aws #cloud #firehose
7
Comments 3
4 min read
Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025
bolewara profile

Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025

#machinelearning #deeplearning #dataengineering #pca
1
Comments
4 min read
Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™
celestehorgan profile

Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™

#postgres #dataengineering #learning #database
1
Comments
10 min read
From smog to streams: how data engineering helps us breathe easier.
oliver_samuel_028c6f65ad6 profile

From smog to streams: how data engineering helps us breathe easier.

#architecture #dataengineering #opensource
1
Comments 1
4 min read
Data Quality at Scale: Why Your Pipeline Needs More Than Green Checkmarks
pradeep_kaalluri profile

Data Quality at Scale: Why Your Pipeline Needs More Than Green Checkmarks

#dataengineering #dataquality #bigdata #python
Comments
8 min read
The Waterfall Pattern: A Tiered Strategy for Reliable Data Extraction
withatte profile

The Waterfall Pattern: A Tiered Strategy for Reliable Data Extraction

#webscraping #dataengineering #devops #dataextraction
1
Comments 1
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.