DEV Community

#bigdata

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Stream 50 million rows without OOM: Lazy chunked Query Streaming.
william_rodriguez_65a5898 profile

Stream 50 million rows without OOM: Lazy chunked Query Streaming.

#bigdata
Comments 1
1 min read
What a Delta table actually is: Parquet files plus a transaction log
dino_david_c5d8f55f119b7f profile

What a Delta table actually is: Parquet files plus a transaction log

#deltalake #databricks #dataengineering #bigdata
Comments
3 min read
Lakehouse vs Data Warehouse vs Data Lake: The Difference in One Picture
dino_david_c5d8f55f119b7f profile

Lakehouse vs Data Warehouse vs Data Lake: The Difference in One Picture

#dataengineering #databricks #deltalake #bigdata
Comments
2 min read
Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization(Part 3 of 3)
zgl20053779 profile

Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization(Part 3 of 3)

#productivity #database #bigdata #datalineage
Comments
2 min read
Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization (Part 2 of 3)
zgl20053779 profile

Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization (Part 2 of 3)

#sql #database #datalineage #bigdata
Comments
4 min read
Pruthivika Meets Redshift: Turning Big Data into Smart Insights ☁️📊
pruthivika_mr_d2457d08566 profile

Pruthivika Meets Redshift: Turning Big Data into Smart Insights ☁️📊

#analytics #aws #bigdata #database
Comments
5 min read
Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization (Part 1 of 3)
zgl20053779 profile

Open-source tool: Cross-database universal "field-level" data lineage analysis and visualization (Part 1 of 3)

#datalineage #sql #bigdata #database
Comments
2 min read
Zero-copy columnar transfer: Apache Arrow meets ClickHouse in Python.
william_rodriguez_65a5898 profile

Zero-copy columnar transfer: Apache Arrow meets ClickHouse in Python.

#bigdata
Comments 1
1 min read
Modern Data Lakehouses: How Apache Iceberg Solved the Pitfalls of Hive Metastore
adhishree_21 profile

Modern Data Lakehouses: How Apache Iceberg Solved the Pitfalls of Hive Metastore

#bigdata #apacheiceberg #dataengineering #dataplatform
Comments
8 min read
Python Pandas to the Rescue: The Ultimate Data Cleanup Guide
arapzruto profile

Python Pandas to the Rescue: The Ultimate Data Cleanup Guide

#pandas #datascience #dataanalytics #bigdata
Comments
10 min read
PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere
nbaubek profile

PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere

#dataengineering #python #bigdata #pyarrow
Comments
10 min read
Apache Data Lakehouse Weekly: September 3-9, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: September 3-9, 2026

#bigdata #database #dataengineering #rust
Comments 1
25 min read
Apache Spark won't melt down when your data doesn't fit in RAM anymore
jtorchia profile

Apache Spark won't melt down when your data doesn't fit in RAM anymore

#english #java #bigdata #spark
1
Comments 1
5 min read
From Pandas DataFrame to ClickHouse in milliseconds without SQL loops.
william_rodriguez_65a5898 profile

From Pandas DataFrame to ClickHouse in milliseconds without SQL loops.

#bigdata
Comments 1
1 min read
Never create 1-row ClickHouse parts again: The built-in Buffer Manager.
william_rodriguez_65a5898 profile

Never create 1-row ClickHouse parts again: The built-in Buffer Manager.

#clickhouse #python #bigdata #dataengineering
Comments 1
1 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.