DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
One Copy, Two Engines: Architecting a Governed Lakehouse Across Fabric and Azure Databricks

One Copy, Two Engines: Architecting a Governed Lakehouse Across Fabric and Azure Databricks

Comments
6 min read
🔐 Storage Credentials in Databricks Unity Catalog (AWS) — The Security Backbone You Can't Ignore

🔐 Storage Credentials in Databricks Unity Catalog (AWS) — The Security Backbone You Can't Ignore

Comments
3 min read
They Gave Me a 20-Hour Take-Home. I Did It. I Got Ghosted.

They Gave Me a 20-Hour Take-Home. I Did It. I Got Ghosted.

Comments
7 min read
How Old Is My Data? The Missing OpenTelemetry Signal

How Old Is My Data? The Missing OpenTelemetry Signal

Comments
5 min read
I Thought My Virtual Environment Was Broken... It Wasn't.

I Thought My Virtual Environment Was Broken... It Wasn't.

Comments
1 min read
RAG Isn’t an AI Problem It’s a Data Pipeline Problem in Disguise

RAG Isn’t an AI Problem It’s a Data Pipeline Problem in Disguise

Comments
3 min read
Data Engineer Salary 2026: Every Survey Is Lying to You

Data Engineer Salary 2026: Every Survey Is Lying to You

Comments
7 min read
I built a terminal SQL workspace for CSV, Parquet, PSV and JSON — and it catches bad data automatically

I built a terminal SQL workspace for CSV, Parquet, PSV and JSON — and it catches bad data automatically

Comments 3
4 min read
Your AI Bill Is a Routing Problem, Not a Model Problem

Your AI Bill Is a Routing Problem, Not a Model Problem

Comments
3 min read
Distributed Metadata Migration

Distributed Metadata Migration

Comments
3 min read
PCI DSS 4.0 Just Made Your Staging Database a Liability. Here Is the Fix

PCI DSS 4.0 Just Made Your Staging Database a Liability. Here Is the Fix

Comments
5 min read
Creating a QuantifiedSelf Dashboard - Part 1

Creating a QuantifiedSelf Dashboard - Part 1

Comments
8 min read
The Model Didn't Change. A Parameter Disappeared. Thousands of Pipelines Broke.

The Model Didn't Change. A Parameter Disappeared. Thousands of Pipelines Broke.

1
Comments
2 min read
Building a right-click 'query this file' workflow in VS Code with DuckDB

Building a right-click 'query this file' workflow in VS Code with DuckDB

Comments
2 min read
The Day I Realized "It Ran Successfully" Means Nothing in Databricks Production

The Day I Realized "It Ran Successfully" Means Nothing in Databricks Production

Comments
6 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.