DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices
robbin_murithi_f75005db58 profile

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

#dataengineering #architecture #tutorial #kafka
Comments
4 min read
A Deep Dive into Apache Spark Architecture
aryan_dandale_e2193e2add3 profile

A Deep Dive into Apache Spark Architecture

#datascience #dataengineering #architecture #performance
Comments
4 min read
Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.
dteimuno profile

Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.

#dataengineering #python #aws #architecture
5
Comments 2
8 min read
Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services
vijaya_bhaskarv_ba95adf9 profile

Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

#dataengineering #performance #architecture #aws
1
Comments
6 min read
What Does a Data Engineer Do? Understanding the Role and Career Path
directdata_education_83e6 profile

What Does a Data Engineer Do? Understanding the Role and Career Path

#beginners #career #dataengineering
Comments
6 min read
Real-Time Data Streaming Platform: From 140K to 1 Million Messages/Sec - A Flink Performance Tuning Journey
vijaya_bhaskarv_ba95adf9 profile

Real-Time Data Streaming Platform: From 140K to 1 Million Messages/Sec - A Flink Performance Tuning Journey

#kubernetes #dataengineering #performance #aws
Comments
10 min read
Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing
yks profile

Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing

#dataengineering #aiops #distributedsystems #machinelearning
10
Comments
6 min read
Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide
vijaya_bhaskarv_ba95adf9 profile

Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

#tutorial #iot #dataengineering #kubernetes
2
Comments
8 min read
Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows
mbessarab profile

Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows

#architecture #automation #dataengineering
Comments
9 min read
Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration
arunkumar_panneerselvam_2 profile

Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration

#dataengineering #cloud #beginners #azure
2
Comments
4 min read
Real-Time Crypto Data Pipeline
lagat_josiah_f024a2c855bc profile

Real-Time Crypto Data Pipeline

#cryptocurrency #dataengineering #architecture #database
Comments
3 min read
Another Data Nerd Guide to re:Invent 2025
elliott_cordo profile

Another Data Nerd Guide to re:Invent 2025

#aws #dataengineering #genai #cloud
1
Comments
3 min read
Real-Time Cryptocurrency Data Pipeline
lagat_josiah_f024a2c855bc profile

Real-Time Cryptocurrency Data Pipeline

#cryptocurrency #dataengineering #performance #architecture
Comments
5 min read
“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”
supraja_tangella_b7f0738e profile

“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”

#dataengineering #etl #elt #azure
5
Comments
1 min read
Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search
apachedoris profile

Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search

#clickhouse #dataengineering #olap #database
5
Comments
22 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.