DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
How to Add a Data Quality Gate to Your Airflow Pipeline in 5 Minutes
appdeviq profile

How to Add a Data Quality Gate to Your Airflow Pipeline in 5 Minutes

#dataengineering #airflow #etl #programming
Comments 1
4 min read
100 Spark Interview Questions for Data Engineer
hannah_usmedynska profile

100 Spark Interview Questions for Data Engineer

#career #dataengineering #interview #resources
1
Comments
11 min read
What Spark Interviews Actually Test (Based on 189 Real Interview Reports)
datadriven profile

What Spark Interviews Actually Test (Based on 189 Real Interview Reports)

#dataengineering #interview #career #programming
Comments
7 min read
Why Semantic Layers Need Distributional Validation, Not Just Schema Validation
anthony_etherealogic profile

Why Semantic Layers Need Distributional Validation, Not Just Schema Validation

#dataquality #semanticlayer #dataengineering #opensource
Comments
9 min read
Bypassing the "Pandas RAM Tax": Building a Zero-Copy CSV Extractor in C
nareshcn2 profile

Bypassing the "Pandas RAM Tax": Building a Zero-Copy CSV Extractor in C

#c #python #dataengineering #performance
1
Comments 1
2 min read
Scuderia Data Ep.3
wvanheemstra profile

Scuderia Data Ep.3

#azure #adf #dataengineering #pipelines
2
Comments
2 min read
Scuderia Data Ep.2
wvanheemstra profile

Scuderia Data Ep.2

#azure #storage #dataengineering #databricks
1
Comments
3 min read
Scuderia Data Ep.1
wvanheemstra profile

Scuderia Data Ep.1

#azure #databricks #dataengineering #beginners
1
Comments
3 min read
Financial Data Integration: A Practical Guide
andrew_tan_layline profile

Financial Data Integration: A Practical Guide

#dataengineering #kafka #eventdriven #fintech
Comments
7 min read
Why Real-Time Data Integration Matters for Modern Applications
andrew_tan_layline profile

Why Real-Time Data Integration Matters for Modern Applications

#data #dataengineering #systemdesign #devops
Comments
5 min read
Why My S3 Backup Setup Broke: Buckets, “Folders”, and Scheduling Misconceptions
mohhddhassan profile

Why My S3 Backup Setup Broke: Buckets, “Folders”, and Scheduling Misconceptions

#aws #s3 #cloudstorage #dataengineering
3
Comments
3 min read
Why Your SAP Migration Mapping Layer Shouldn't Live in Spreadsheets
rationalxplorer profile

Why Your SAP Migration Mapping Layer Shouldn't Live in Spreadsheets

#datamigration #sap #dataengineering #opensource
2
Comments
3 min read
How I Built an End-to-End Data Engineering Pipeline for Hong Kong's Public Transport Network
mfarizala profile

How I Built an End-to-End Data Engineering Pipeline for Hong Kong's Public Transport Network

#dataengineering #datazoomcamp #googlecloud #bigquery
1
Comments
8 min read
Debugging a Broken Metrics Pipeline: What Actually Went Wrong
mohhddhassan profile

Debugging a Broken Metrics Pipeline: What Actually Went Wrong

#dataengineering #clickhouse #vector #debugging
3
Comments
3 min read
Building Consistent Data Foundations at Scale
supratipb profile

Building Consistent Data Foundations at Scale

#data #dataengineering #datascience #database
1
Comments 1
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.