DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
How I Built a MongoDB Archiving System for Crawled Data
pradippanjiyar profile

How I Built a MongoDB Archiving System for Crawled Data

#dataengineering #database #architecture #mongodb
1
Comments 2
7 min read
Complete Guide: Dockerizing Spark, Kafka, and Jupyter for YouTube Pipeline
lagat_josiah_f024a2c855bc profile

Complete Guide: Dockerizing Spark, Kafka, and Jupyter for YouTube Pipeline

#dataengineering #architecture #docker #tutorial
Comments
9 min read
Dockerized Spark and Kafka: YouTube Data Pipeline Implementation
lagat_josiah_f024a2c855bc profile

Dockerized Spark and Kafka: YouTube Data Pipeline Implementation

#dataengineering #architecture #docker #tutorial
Comments
7 min read
RIP Amazon Data Firehose Change Data Capture
psantus profile

RIP Amazon Data Firehose Change Data Capture

#dataengineering #aws #cloud #firehose
7
Comments 3
4 min read
Event-Driven Architectures on AWS: Beyond Lambda
nidhi_0105 profile

Event-Driven Architectures on AWS: Beyond Lambda

#blog #aws #dataengineering #lambda
4
Comments
2 min read
YouTube Data Processing Pipeline
lagat_josiah_f024a2c855bc profile

YouTube Data Processing Pipeline

#postgres #python #dataengineering #architecture
2
Comments 1
4 min read
🔄 ETL vs ELT: What’s the Difference and Why It Matters?
shagun_khandelwal profile

🔄 ETL vs ELT: What’s the Difference and Why It Matters?

#dataengineering #python #sql #datawarehouse
Comments
2 min read
CDC in AWS: Content Data Capture from AWS RDS MySQL into AWS MSK Kafka topic using Debezium
tsypuk profile

CDC in AWS: Content Data Capture from AWS RDS MySQL into AWS MSK Kafka topic using Debezium

#dataengineering #mysql #aws #kafka
1
Comments
5 min read
LLPY-03: Extracción y Procesamiento Inteligente de Datos Legales
jesus_oviedoriquelme_084 profile

LLPY-03: Extracción y Procesamiento Inteligente de Datos Legales

#dataengineering #spanish #rag #llm
Comments
21 min read
🏗️ The Role of a Data Engineer: Beyond Pipelines
parth_maniar_3012 profile

🏗️ The Role of a Data Engineer: Beyond Pipelines

#datascience #dataengineering #bigdata #career
Comments
2 min read
Beyond Flat Tables: Model Hierarchical Data in Supabase with Recursive Queries
roel_peters_8b77a70a08fdb profile

Beyond Flat Tables: Model Hierarchical Data in Supabase with Recursive Queries

#postgres #database #fullstack #dataengineering
2
Comments
7 min read
🌍 The Journey of Data: From Raw Logs to Insights
shagun_khandelwal profile

🌍 The Journey of Data: From Raw Logs to Insights

#dataengineering #bigdata #etl #python
Comments
2 min read
Git Integration in Microsoft Fabric
anshul_02 profile

Git Integration in Microsoft Fabric

#dataengineering #fabric #microsoft
4
Comments
3 min read
🎯 The Challenge: Processing TBs of S3 Data Without Breaking the Bank
qvfagundes profile

🎯 The Challenge: Processing TBs of S3 Data Without Breaking the Bank

#data #dataengineering #ai
Comments
5 min read
Why Apache Iceberg is needed?
minwook profile

Why Apache Iceberg is needed?

#dataengineering #ai #programming #robotics
1
Comments
6 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.