Dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

AlaiKrm

Jun 10

The Data Ingestion Pipeline Nobody Designs Well Until Production Breaks It

#ai #dataengineering #rag #systemdesign

5 min read

Matheus Dallacort

Jun 10

Modern Data Stack Migration — Day 1: Scaling to 8+ Companies with DRY Architecture and Chasing a $2M Discrepancy

#dataengineering #python #dbt #architecture

2 min read

Haji Rufai

Jun 13

I Built a Token-Bucket Rate Limiter in Pure Python and Finally Understood How APIs Protect Themselves

#python #dataengineering #programming #tutorial

6 min read

Haji Rufai

Jun 10

I built a feature store in pure Python to finally understand the point-in-time join

#python #machinelearning #dataengineering #mlops

6 min read

Vaibhav

Jul 14

Building an AI Claims-Adjuster Agent: Architecture, Guardrails, and the Data It Actually Needs

#insurtech #ai #dataengineering #machinelearning

4 min read

Vaibhav

Jul 14

Why 80% of Insurance AI Projects Stall — And It's Not the Model

#insurtech #ai #dataengineering #data

3 min read

Alex Merced

Jul 13

A Deep Dive Into File Compression: How Data Gets Smaller, Why Codecs Differ, and What to Actually Use in the Lakehouse

#data #dataengineering #infrastructure #performance

20 min read

Alex Merced

Jul 14

File Encryption for the Lakehouse: The Terminology, the Machinery, and the Hard Problem of Interoperable Encrypted Tables

#database #dataengineering #opensource #security

20 min read

Alex Merced

Jul 13

A Reader's Guide to My Books: Which One to Pick Up, Depending on What You're Building

#ai #dataengineering #learning #softwaredevelopment

11 min read

Haji Rufai

Jun 9

I Built a Bloom Filter from Scratch in Pure Python and Finally Understood How Databases Skip Reading Data

#python #dataengineering #database #programming

6 min read

Firat Celik

Jun 9

# go-intake: Go-Native Streaming Data Ingestion Toolkit

#etl #dataengineering #go #opensource

3 min read

Paulet Wairagu

Jun 9

QN : Ingest Data with Dataflows Gen2 in Microsoft Fabric

#data #fabric #dataengineering

2 min read

Bob Oner

Jun 9

From Data Quality Checks to Analytics-Ready Parquet with Python

#python #etl #dataengineering #docker

8 min read

Alex Merced

Jul 13

Lakehouse Table Formats in 2026: Iceberg, Delta Lake, Hudi, Paimon, and DuckLake, How They Work, Where They Stand, and Where They're Going

#architecture #database #dataengineering #opensource

22 min read

Alex Merced

Jul 13

The File Format Renaissance: Parquet, Lance, Vortex, Nimble, BtrBlocks, and the New Physics of Columnar Storage

#architecture #database #dataengineering #machinelearning

21 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.