Skip to content
Navigation menu
Search
Powered by Algolia
Search
Log in
Create account
DEV Community
Close
#
dataengineering
Follow
Hide
Posts
Left menu
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Advanced Kubernetes Patterns for Data Engineers
Victor Kipruto
Victor Kipruto
Victor Kipruto
Follow
Jun 8
Advanced Kubernetes Patterns for Data Engineers
#
kubernetes
#
dataengineering
#
docker
#
devops
5
 reactions
Comments
Add Comment
1 min read
Moderation
Extract data from Databases into DuckLake
Fritz Larco
Fritz Larco
Fritz Larco
Follow
Jun 8
Extract data from Databases into DuckLake
#
duckdb
#
ducklake
#
dataengineering
#
etl
Comments
Add Comment
4 min read
Moderation
How I forced Python standard libraries to process and serialize production server crashes into Parquet locally
Syed Amman qadir bukhari
Syed Amman qadir bukhari
Syed Amman qadir bukhari
Follow
Jun 8
How I forced Python standard libraries to process and serialize production server crashes into Parquet locally
#
python
#
dataengineering
#
devops
#
opensource
Comments
Add Comment
1 min read
Moderation
From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering
Jitendra Devabhaktuni
Jitendra Devabhaktuni
Jitendra Devabhaktuni
Follow
Jun 8
From Clean CSVs to Production‑Shaped Data: A Practical Guide for Academic ML and Data Engineering
#
ai
#
database
#
datascience
#
dataengineering
Comments
Add Comment
5 min read
Moderation
Data Engineering: The Data Field's Unknown Child
Itoro James
Itoro James
Itoro James
Follow
Jul 12
Data Engineering: The Data Field's Unknown Child
#
beginners
#
career
#
data
#
dataengineering
3
 reactions
Comments
Add Comment
4 min read
Moderation
I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes
Haji Rufai
Haji Rufai
Haji Rufai
Follow
Jun 8
I Built a Write-Ahead Log in Pure Python and Finally Understood How Databases Survive Crashes
#
python
#
database
#
dataengineering
#
programming
Comments
Add Comment
7 min read
Moderation
Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500
Pratik Dhanave
Pratik Dhanave
Pratik Dhanave
Follow
Jun 8
Architecture Over Alerts: How We Cut BigQuery Costs by 57%($12M) for a Fortune 500
#
bigquery
#
finops
#
cloudarchitecture
#
dataengineering
Comments
Add Comment
4 min read
Moderation
I Built a Columnar File Format in Pure Python — a tiny, readable Parquet
Haji Rufai
Haji Rufai
Haji Rufai
Follow
Jun 8
I Built a Columnar File Format in Pure Python — a tiny, readable Parquet
#
python
#
dataengineering
#
database
#
programming
Comments
Add Comment
6 min read
Moderation
Running Apache Airflow + Docker for Free Using GitHub Codespaces
Tanmay
Tanmay
Tanmay
Follow
Jun 8
Running Apache Airflow + Docker for Free Using GitHub Codespaces
#
dataengineering
#
docker
#
python
#
apacheairflow
Comments
Add Comment
1 min read
Moderation
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
Alex Merced
Alex Merced
Alex Merced
Follow
Jul 12
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
#
architecture
#
data
#
database
#
dataengineering
Comments
1
 comment
21 min read
Moderation
Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast
Alex Merced
Alex Merced
Alex Merced
Follow
Jul 12
Block vs. Object Storage: A Deep Dive Into the Foundation of Modern Data, and How the Lakehouse Made the Slow Option Fast
#
architecture
#
dataengineering
#
infrastructure
#
performance
2
 reactions
Comments
1
 comment
22 min read
Moderation
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
Gathuru_M
Gathuru_M
Gathuru_M
Follow
Jun 7
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
#
dataengineering
#
api
#
etl
#
beginners
1
 reaction
Comments
Add Comment
4 min read
Moderation
The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted
Alex Merced
Alex Merced
Alex Merced
Follow
Jul 12
The Who, What, and Why of Semantic Layers: The Layer That Decides Whether Your Numbers Can Be Trusted
#
ai
#
analytics
#
data
#
dataengineering
Comments
Add Comment
22 min read
Moderation
I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast
Haji Rufai
Haji Rufai
Haji Rufai
Follow
Jun 7
I built a mini columnar storage engine in pure Python — and finally understood why Parquet is so fast
#
dataengineering
#
python
#
database
#
performance
Comments
Add Comment
6 min read
Moderation
Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables
Aki
Aki
Aki
Follow
for
AWS Community Builders
Jul 10
Hitting the Iceberg REST Catalog Directly: Understanding the Differences Between Glue Data Catalog and S3 Tables
#
aws
#
iceberg
#
dataengineering
9
 reactions
Comments
2
 comments
17 min read
Moderation
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account