DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
From Detection to Production: A PII-Safe Pipeline in Python and DuckDB

From Detection to Production: A PII-Safe Pipeline in Python and DuckDB

Comments
6 min read
You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines

You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines

Comments
8 min read
I Built a Data Quality Library Because Existing Tools Were Too Slow for My Pipelines

I Built a Data Quality Library Because Existing Tools Were Too Slow for My Pipelines

Comments 1
3 min read
Replace Fivetran + Snowflake: MySQL to S3 Parquet Analytics, Self-Hosted

Replace Fivetran + Snowflake: MySQL to S3 Parquet Analytics, Self-Hosted

2
Comments
6 min read
I filtered the FMCSA census (4.5M rows) down to carriers that look real

I filtered the FMCSA census (4.5M rows) down to carriers that look real

1
Comments 11
2 min read
Apache Data Lakehouse Weekly: September 3-9, 2026

Apache Data Lakehouse Weekly: September 3-9, 2026

Comments 1
25 min read
Avro schemas for Kafka developers: the 10-minute practical guide

Avro schemas for Kafka developers: the 10-minute practical guide

Comments
2 min read
Batch ETL vs Real-Time ETL Tools: Which One Your Team Needs

Batch ETL vs Real-Time ETL Tools: Which One Your Team Needs

13
Comments 1
7 min read
SnowPro Core COF-C03 vs COF-C02: What Changed

SnowPro Core COF-C03 vs COF-C02: What Changed

Comments 1
2 min read
My RAG pipeline died three nights in a row. So I rebuilt it to assume it would die.

My RAG pipeline died three nights in a row. So I rebuilt it to assume it would die.

Comments
2 min read
The hardest label in our dataset is the one where nothing happened

The hardest label in our dataset is the one where nothing happened

Comments
14 min read
Estrategias de Migración de Bases de Datos Relacionales: De AWS RDS a GCP Cloud SQL

Estrategias de Migración de Bases de Datos Relacionales: De AWS RDS a GCP Cloud SQL

1
Comments
6 min read
Escalabilidad Geográfica: Implementación de Sistemas SQL Distribuidos con YugabyteDB

Escalabilidad Geográfica: Implementación de Sistemas SQL Distribuidos con YugabyteDB

Comments
5 min read
Gestión de APIs Empresariales: Integrando Apigee, MuleSoft y Salesforce Data Cloud

Gestión de APIs Empresariales: Integrando Apigee, MuleSoft y Salesforce Data Cloud

Comments
5 min read
FinOps y Gestión del Ciclo de Vida: Desmantelamiento Seguro de Infraestructura Serverless en GCP

FinOps y Gestión del Ciclo de Vida: Desmantelamiento Seguro de Infraestructura Serverless en GCP

Comments
5 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.