DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere
nbaubek profile

PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere

#dataengineering #python #bigdata #pyarrow
Comments
10 min read
Databricks vs Snowflake: Two Different Philosophies for the Same Data Future
laura_williams profile

Databricks vs Snowflake: Two Different Philosophies for the Same Data Future

#databricks #snowflake #dataengineering #cloud
Comments
5 min read
Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases.
gacherimutua profile

Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases.

#dataengineering #airflow
Comments
3 min read
Mask, Hash, Tokenize, or Encrypt? Choosing the Right PII Protection for Your Pipeline
nbaubek profile

Mask, Hash, Tokenize, or Encrypt? Choosing the Right PII Protection for Your Pipeline

#dataengineering #security #dataprivacy #gdpr
Comments
6 min read
From Detection to Production: A PII-Safe Pipeline in Python and DuckDB
nbaubek profile

From Detection to Production: A PII-Safe Pipeline in Python and DuckDB

#dataengineering #duckdb #dataprivacy #security
Comments
6 min read
You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines
nbaubek profile

You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines

#dataengineering #python #dataprivacy #gdpr
Comments
8 min read
I Built a Data Quality Library Because Existing Tools Were Too Slow for My Pipelines
vladislav_koval_f4bed0e76 profile

I Built a Data Quality Library Because Existing Tools Were Too Slow for My Pipelines

#dataengineering #python #dataquality #bigquery
Comments 1
3 min read
Apache Data Lakehouse Weekly: September 3-9, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: September 3-9, 2026

#bigdata #database #dataengineering #rust
Comments 1
25 min read
Avro schemas for Kafka developers: the 10-minute practical guide
yuus_company profile

Avro schemas for Kafka developers: the 10-minute practical guide

#kafka #avro #dataengineering #tutorial
Comments
2 min read
Ad Measurement Loses Events at Peak: Seams, Duplicate Keys, and the Bidder Join
ambrain profile

Ad Measurement Loses Events at Peak: Seams, Duplicate Keys, and the Bidder Join

#kafka #clickhouse #dataengineering #adtech
Comments 1
9 min read
The hardest label in our dataset is the one where nothing happened
inve_money_e95961f8489d28 profile

The hardest label in our dataset is the one where nothing happened

#datascience #dataengineering #analytics #sql
Comments
14 min read
Recursive CTEs, explained from first principles
sophiemwangi profile

Recursive CTEs, explained from first principles

#sql #cte #dataengineering #datascience
3
Comments 2
5 min read
I Analyzed 2,061 Public n8n Workflows and Found 92% of the AI Nodes Had Been Silently Replaced
adix65 profile

I Analyzed 2,061 Public n8n Workflows and Found 92% of the AI Nodes Had Been Silently Replaced

#ai #opensource #dataengineering #automation
Comments
4 min read
Azure Synapse Databricks: A Migration Playbook That Survives Production
firfircelik profile

Azure Synapse Databricks: A Migration Playbook That Survives Production

#databricks #azure #dataengineering #etl
Comments
9 min read
Best Dimensional Data Modeling Tools: erwin Data Modeler Features & Alternatives
rutadbm profile

Best Dimensional Data Modeling Tools: erwin Data Modeler Features & Alternatives

#data #database #dataengineering #tools
Comments
11 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.