Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Aimé Bangirahe

Nov 10 '25

Kafka consumer lag—Measure and reduce

#devops #performance #dataengineering #monitoring

5 min read

Eric Kahindi

Nov 10 '25

Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It

#systemdesign #performance #dataengineering #monitoring

3 min read

remis haroon

Dec 15 '25

Building a dbt-UI I Wish Existed

#dbt #data #dataengineering #dataanalytics

3 min read

Aineah Simiyu

Nov 10 '25

Building a Real-Time Crypto Data Pipeline with Debezium CDC

#python #kafka #cdc #dataengineering

5 min read

John Kioko

Nov 10 '25

Undestanding Kafka Lag, Why It Happens and How To Fix It.

#kafka #programming #datascience #dataengineering

4 min read

Judy

Dec 15 '25

The Myth of Distributed Computing as a Silver Bullet for Big Data

#discuss #dataengineering #performance #architecture

10 min read

Alex Merced

Nov 11 '25

The State of Apache Iceberg, Polaris, and Arrow: November 5-11

#opensource #database #dataengineering #data

5 min read

Oliver Samuel

Nov 10 '25

Understanding Kafka Lag: Why It Happens and How to Fix It

#dataengineering #monitoring #performance

4 min read

ObservabilityGuy

Nov 10 '25

Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS

#analytics #cloud #tutorial #dataengineering

7 min read

robbin murithi

Nov 10 '25

Understanding reasons behind Kafka lag and how to minimize it.

#dataengineering #monitoring #performance

3 min read

Chetan Gupta

Dec 14 '25

REST API Calls for Data Engineers: A Practical Guide with Examples

#discuss #rest #dataengineering #json

3 min read

Chetan Gupta

Dec 14 '25

Why Idempotence Is So Important in Data Engineering

#architecture #dataengineering #systemdesign #etl

6 min read

Aineah Simiyu

Nov 10 '25

Reducing Consumer Lag in Apache Kafka

#kafka #dataengineering #python #eventdriven

3 min read

Mohamed Hussain S

Dec 13 '25

Analyzing and Optimizing a Parquet ClickHouse Ingestion Pipeline

#dataengineering #etl #clickhouse #python

3 min read

Sandeep

Dec 1 '25

🚀 Day 1: Introduction to Apache Spark

#spark #python #dataengineering #bigdata

2 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# dataengineering

Kafka consumer lag—Measure and reduce

Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It

Building a dbt-UI I Wish Existed

Building a Real-Time Crypto Data Pipeline with Debezium CDC

Undestanding Kafka Lag, Why It Happens and How To Fix It.

The Myth of Distributed Computing as a Silver Bullet for Big Data

The State of Apache Iceberg, Polaris, and Arrow: November 5-11

Understanding Kafka Lag: Why It Happens and How to Fix It

Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS

Understanding reasons behind Kafka lag and how to minimize it.

REST API Calls for Data Engineers: A Practical Guide with Examples

Why Idempotence Is So Important in Data Engineering

Reducing Consumer Lag in Apache Kafka

Analyzing and Optimizing a Parquet ClickHouse Ingestion Pipeline

🚀 Day 1: Introduction to Apache Spark