DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering
jitendra_devabhaktuni_0f1 profile

From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering

#ai #database #datascience #dataengineering
Comments
5 min read
Data Engineering: The Data Field's Unknown Child
khaytorah profile

Data Engineering: The Data Field's Unknown Child

#beginners #career #data #dataengineering
3
Comments
4 min read
I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes
hajirufai profile

I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes

#python #database #dataengineering #programming
Comments
7 min read
Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500
pratikdhanave profile

Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500

#bigquery #finops #cloudarchitecture #dataengineering
Comments
4 min read
I Built a Columnar File Format in Pure Python — a tiny, readable Parquet
hajirufai profile

I Built a Columnar File Format in Pure Python — a tiny, readable Parquet

#python #dataengineering #database #programming
Comments
6 min read
Running Apache Airflow + Docker for Free Using GitHub Codespaces
tanmay_bhurkunde profile
Tanmay

Running Apache Airflow + Docker for Free Using GitHub Codespaces

#dataengineering #docker #python #apacheairflow
Comments
1 min read
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
alexmercedcoder profile

Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take

#architecture #data #database #dataengineering
Comments 1
21 min read
Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast
alexmercedcoder profile

Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast

#architecture #dataengineering #infrastructure #performance
2
Comments 1
22 min read
The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted
alexmercedcoder profile

The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted

#ai #analytics #data #dataengineering
Comments
22 min read
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
gathurum profile

ETL Pipeline: Fetching Real-Time News Data with Python and Postgres

#dataengineering #api #etl #beginners
1
Comments
4 min read
I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast
hajirufai profile

I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast

#dataengineering #python #database #performance
Comments
6 min read
Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables
datapenguin profile

Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables

#aws #iceberg #dataengineering
9
Comments 2
17 min read
Why I Don’t Let the LLM Decide Issue State
ahmadhumayun profile

Why I Don’t Let the LLM Decide Issue State

#ai #python #dataengineering #marketinganalytics
Comments
5 min read
The customer_id that isn't a customer
gaur_data_c6511af396c4bce profile

The customer_id that isn't a customer

#agents #analytics #dataengineering #sql
Comments
8 min read
Your Scraper Collected 50 Rows. There Were 4,000.
0012303 profile

Your Scraper Collected 50 Rows. There Were 4,000.

#webscraping #python #dataengineering #pagination
Comments
7 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.