DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)
alexmercedcoder profile

Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)

#data #dataengineering #opensource #resources
Comments
5 min read
How to Sync Data from an Oracle Table to Elasticsearch using Kafka Connect
maghsood_esmaeili profile

How to Sync Data from an Oracle Table to Elasticsearch using Kafka Connect

#database #kubernetes #dataengineering #tutorial
1
Comments 1
5 min read
From Raw to Refined: Data Pipeline Architecture at Scale
pradeep_kaalluri profile

From Raw to Refined: Data Pipeline Architecture at Scale

#dataengineering #bigdata #python #dataquality
Comments
12 min read
Agent Cost Optimization: A Data Engineer's Guide
hoshang_mehta profile

Agent Cost Optimization: A Data Engineer's Guide

#dataengineering #ai
Comments
13 min read
INTRODUCTION TO DBT(Data Build Tool)
john_analytics profile

INTRODUCTION TO DBT(Data Build Tool)

#dataengineering #data #analytics
1
Comments
2 min read
What's New in dbt 1.10
shammadahmed profile

What's New in dbt 1.10

#dbt #data #datascience #dataengineering
3
Comments
12 min read
The Day Our Pipeline Went From 10 Minutes to 6 Seconds (Part 2 of 3)
karthikeyan_rajasekaran_c profile

The Day Our Pipeline Went From 10 Minutes to 6 Seconds (Part 2 of 3)

#dataengineering #dbt #dagster #data
2
Comments
7 min read
How Strategic Image Cropping Transforms Data Ingestion Pipelines
steravy profile

How Strategic Image Cropping Transforms Data Ingestion Pipelines

#dataengineering #machinelearning #performance
1
Comments
4 min read
Taming the Data Beast: Build Pipelines That Bend, Not Break by Arvind Sundararajan
arvind_sundararajan profile

Taming the Data Beast: Build Pipelines That Bend, Not Break by Arvind Sundararajan

#dataengineering #datascience #architecture #data
Comments
2 min read
Snowflake + Postgres: A Small Feature That Signals a Big Shift
shrinivasv73 profile

Snowflake + Postgres: A Small Feature That Signals a Big Shift

#snowflake #postgres #database #dataengineering
Comments
6 min read
Data Quality at Scale: Why Your Pipeline Needs More Than Green Checkmarks
pradeep_kaalluri profile

Data Quality at Scale: Why Your Pipeline Needs More Than Green Checkmarks

#dataengineering #dataquality #bigdata #python
Comments
8 min read
Prompt Engineering Patterns: From Zero-Shot to Chain-of-Thought Reasoning
qvfagundes profile

Prompt Engineering Patterns: From Zero-Shot to Chain-of-Thought Reasoning

#ai #mcp #llm #dataengineering
1
Comments
14 min read
Why We Need Schema Registry in Kafka
jumptotech profile

Why We Need Schema Registry in Kafka

#systemdesign #distributedsystems #dataengineering #architecture
2
Comments
17 min read
Introduction to the Confluent REST Proxy
jumptotech profile

Introduction to the Confluent REST Proxy

#api #architecture #dataengineering
2
Comments
4 min read
While We're Measuring Developer Productivity, Won't Someone Think of the Data Engineers?
aronchick profile

While We're Measuring Developer Productivity, Won't Someone Think of the Data Engineers?

#dataengineering #devrel #productivity #observability
Comments
9 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.