DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
The Data Engineer’s Codex: From First Principles to the Modern Lakehouse
datatechbridge profile

The Data Engineer’s Codex: From First Principles to the Modern Lakehouse

#systemdesign #distributedsystems #dataengineering #architecture
6
Comments
10 min read
Breaking Into Gaming Analytics: From 1 Billion Mobile Users to 5B Daily Events
sai_krishnachaitanyachi profile

Breaking Into Gaming Analytics: From 1 Billion Mobile Users to 5B Daily Events

#dataengineering #analytics #gaminganalytics #careerpivot
Comments 1
6 min read
Building a Real-Time Data Lake on AWS: S3, Glue, and Athena in Production
wanjohichristopher profile

Building a Real-Time Data Lake on AWS: S3, Glue, and Athena in Production

#architecture #aws #dataengineering
1
Comments
5 min read
Embeddings and Vector Similarity: How Machines Understand Meaning
qvfagundes profile

Embeddings and Vector Similarity: How Machines Understand Meaning

#ai #datascience #dataengineering #llm
1
Comments
19 min read
Containerization for Data Engineering: A Practical Guide with Docker and Docker Compose
j_m47 profile

Containerization for Data Engineering: A Practical Guide with Docker and Docker Compose

#docker #dataengineering #devops #containers
Comments
2 min read
Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)
altinity profile

Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)

#events #opensource #ai #dataengineering
Comments
3 min read
AWS Glue for ETL
godofgeeks profile

AWS Glue for ETL

#aws #dataengineering #serverless
Comments
5 min read
What to use for data preparation in report, query or analysis business?
esproc_spl profile

What to use for data preparation in report, query or analysis business?

#analytics #dataengineering #datascience
5
Comments
10 min read
Optimizing Data Processing on AWS with Data Compaction
datapenguin profile

Optimizing Data Processing on AWS with Data Compaction

#aws #dataengineering #iceberg #lambda
4
Comments
7 min read
Real-Time Earthquake CDC Pipeline
oliver_samuel_028c6f65ad6 profile

Real-Time Earthquake CDC Pipeline

#architecture #database #dataengineering
Comments
5 min read
Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS
datapenguin profile

Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS

#aws #dataengineering #lambda #sqs
3
Comments
6 min read
The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
markolekic profile

The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane

#python #dataengineering #api #sql
4
Comments
5 min read
Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices
jumptotech profile

Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices

#architecture #dataengineering #microservices #beginners
2
Comments
3 min read
Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025
bolewara profile

Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025

#machinelearning #deeplearning #dataengineering #pca
1
Comments
4 min read
An Exploration of the Commercial Iceberg Catalog Ecosystem
alexmercedcoder profile

An Exploration of the Commercial Iceberg Catalog Ecosystem

#dataengineering
Comments
14 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.