DEV Community

# bigdata

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Beyond Tagging: A Blueprint for Real-Time Cost Attribution in Data Platforms
mahendran_a3347c2dfcca93e profile

Beyond Tagging: A Blueprint for Real-Time Cost Attribution in Data Platforms

#dataengineering #finops #bigdata #costoptimization
Comments
9 min read
Day 16: Delta Lake Explained - How Spark Finally Became Reliable for Production ETL
sandeepk27 profile

Day 16: Delta Lake Explained - How Spark Finally Became Reliable for Production ETL

#python #dataengineering #spark #bigdata
Comments
2 min read
Day 15: Running Spark in the Cloud - Dataproc vs Databricks
sandeepk27 profile

Day 15: Running Spark in the Cloud - Dataproc vs Databricks

#python #dataengineering #spark #bigdata
Comments
2 min read
Day 14: Building a Real Retail Analytics Pipeline Using Spark Window Functions
sandeepk27 profile

Day 14: Building a Real Retail Analytics Pipeline Using Spark Window Functions

#python #dataengineering #spark #bigdata
Comments
1 min read
Day 13: Window Functions in PySpark
sandeepk27 profile

Day 13: Window Functions in PySpark

#python #dataengineering #spark #bigdata
Comments
2 min read
Day 17: Building a Real ETL Pipeline in Spark Using Bronze-Silver-Gold Architecture
sandeepk27 profile

Day 17: Building a Real ETL Pipeline in Spark Using Bronze-Silver-Gold Architecture

#dataengineering #spark #bigdata #python
Comments
1 min read
Day 12: UDF vs Pandas UDF
sandeepk27 profile

Day 12: UDF vs Pandas UDF

#python #dataengineering #spark #bigdata
Comments
2 min read
Agent Facing Analytics with High Concurrency: Doris vs Clickhouse vs Snowflake
apachedoris profile

Agent Facing Analytics with High Concurrency: Doris vs Clickhouse vs Snowflake

#bigdata #ai #apachedoris #database
Comments
5 min read
Connector Fixes, Core API Enhancements, and Ecosystem Updates: Apache SeaTunnel’s Progress in November
seatunnel profile

Connector Fixes, Core API Enhancements, and Ecosystem Updates: Apache SeaTunnel’s Progress in November

#apacheseatunnel #development #bigdata #datascience
Comments
6 min read
Day 11: Choosing the Right File Format in Spark
sandeepk27 profile

Day 11: Choosing the Right File Format in Spark

#python #dataengineering #spark #bigdata
Comments
2 min read
From Bug Fixes to Ecosystem Enhancements: Key Highlights from DolphinScheduler’s November Updates
chen_debra_3060b21d12b1b0 profile

From Bug Fixes to Ecosystem Enhancements: Key Highlights from DolphinScheduler’s November Updates

#apachedolphinscheduler #opensource #bigdata #development
Comments
5 min read
Day 7: Mastering Joins, Unions, and GroupBy in PySpark - The Core ETL Operations
sandeepk27 profile

Day 7: Mastering Joins, Unions, and GroupBy in PySpark - The Core ETL Operations

#dataengineering #python #spark #bigdata
Comments
2 min read
2025 Year in Review: Apache Iceberg, Polaris, Parquet, and Arrow
alexmercedcoder profile

2025 Year in Review: Apache Iceberg, Polaris, Parquet, and Arrow

#architecture #bigdata #opensource #dataengineering
Comments
6 min read
Day 9: Spark SQL Deep Dive - Temp Views, Query Execution & Optimization Tips for Data Engineers
sandeepk27 profile

Day 9: Spark SQL Deep Dive - Temp Views, Query Execution & Optimization Tips for Data Engineers

#python #dataengineering #spark #bigdata
Comments
2 min read
Day 10: Partitioning vs Bucketing - The Spark Optimization Guide Every Data Engineer Needs
sandeepk27 profile

Day 10: Partitioning vs Bucketing - The Spark Optimization Guide Every Data Engineer Needs

#python #dataengineering #spark #bigdata
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.