DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
CSV: The Format Nobody Designed
vivian-voss profile

CSV: The Format Nobody Designed

#csv #dataengineering #unix #architecture
Comments
2 min read
I Analyzed 10 Million Records in 47 Seconds Using Python + DuckDB (No Spark, No Cloud)
dattasable profile

I Analyzed 10 Million Records in 47 Seconds Using Python + DuckDB (No Spark, No Cloud)

#sql #python #dataengineering #tutorial
3
Comments 2
3 min read
We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed
tasrieitservices profile

We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed

#airflow #dataengineering #kubernetes #eks
1
Comments
12 min read
Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job
luca1iu profile
Jin

Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job

#azure #dataanalyst #functions #dataengineering
Comments
3 min read
CTEs vs Subqueries in SQL: Which One Should You Use?
neha_christina_1ac8651819 profile

CTEs vs Subqueries in SQL: Which One Should You Use?

#sql #dataengineering #beginners #programming
Comments
3 min read
Augmented Analytics: AI-Assisted Data Insights and the Future of Intelligent Decision-Making
vishaluttammane profile

Augmented Analytics: AI-Assisted Data Insights and the Future of Intelligent Decision-Making

#dataengineering #analytics #aiops #machinelearning
1
Comments 1
2 min read
SQL String Functions: Clean, Transform, and Tame Messy Data Like a Pro
vivekdraxlr profile

SQL String Functions: Clean, Transform, and Tame Messy Data Like a Pro

#sql #database #postgres #dataengineering
Comments
6 min read
How to Optimize BigQuery Costs (Real Techniques That Work)
amaranarun profile

How to Optimize BigQuery Costs (Real Techniques That Work)

#bigquery #dataengineering #googlecloud #sql
1
Comments
9 min read
🌐 Beyond One Data Source: Building Scalable Data Pipelines in Power BI
abooafifah profile

🌐 Beyond One Data Source: Building Scalable Data Pipelines in Power BI

#analytics #data #dataengineering #tutorial
Comments
5 min read
ETL vs ELT: The Difference That Finally Clicked for Me
nandaha profile

ETL vs ELT: The Difference That Finally Clicked for Me

#datascience #dataengineering #etl #elt
1
Comments
2 min read
Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained
gracemunyi profile

Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained

#discuss #ai #beginners #dataengineering
Comments
8 min read
AI Broke Data Engineering Interviews. Nobody Knows What's Next.
datadriven profile

AI Broke Data Engineering Interviews. Nobody Knows What's Next.

#dataengineering #interview #ai #career
Comments
7 min read
AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns
datapenguin profile

AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns

#aws #snowflake #dataengineering #apacheiceberg
5
Comments
7 min read
Backfilling Is Harder Than Scraping: Lessons From Rebuilding 6 Months of Missing Data
anna_6c67c00f5c3f53660978 profile
Anna

Backfilling Is Harder Than Scraping: Lessons From Rebuilding 6 Months of Missing Data

#webscraping #dataengineering #rapidproxy #architecture
Comments
3 min read
Designing Stable Integration Testing Architectures for Data-Driven Systems By QA Transformation & Integration Architect
chirag_patel_da672dcd5a8e profile

Designing Stable Integration Testing Architectures for Data-Driven Systems By QA Transformation & Integration Architect

#opensource #automation #azure #dataengineering
Comments
3 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.