DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
The real problem with ingesting MongoDB into Delta Lake (and how I built a library to fix it)
lholiv profile

The real problem with ingesting MongoDB into Delta Lake (and how I built a library to fix it)

#dataengineering #python #opensource #mongodb
2
Comments 7
5 min read
CSV: The Format Nobody Designed
vivian-voss profile

CSV: The Format Nobody Designed

#csv #dataengineering #unix #architecture
Comments
2 min read
We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed
tasrieitservices profile

We Upgraded Airflow 2.8 to 3.1 on Kubernetes. Here Is What Actually Changed

#airflow #dataengineering #kubernetes #eks
1
Comments
12 min read
Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job
luca1iu profile
Jin

Stop Using Spark for Your Small Data - Why Azure Functions is the Right Tool for the Job

#azure #dataanalyst #functions #dataengineering
Comments
3 min read
CTEs vs Subqueries in SQL: Which One Should You Use?
neha_christina_1ac8651819 profile

CTEs vs Subqueries in SQL: Which One Should You Use?

#sql #dataengineering #beginners #programming
Comments
3 min read
Augmented Analytics: AI-Assisted Data Insights and the Future of Intelligent Decision-Making
vishaluttammane profile

Augmented Analytics: AI-Assisted Data Insights and the Future of Intelligent Decision-Making

#dataengineering #analytics #aiops #machinelearning
1
Comments 1
2 min read
🌐 Beyond One Data Source: Building Scalable Data Pipelines in Power BI
abooafifah profile

🌐 Beyond One Data Source: Building Scalable Data Pipelines in Power BI

#analytics #data #dataengineering #tutorial
Comments
5 min read
Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained
gracemunyi profile

Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained

#discuss #ai #beginners #dataengineering
Comments
8 min read
AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns
datapenguin profile

AWS Snowflake Lakehouse: 2 Practical Apache Iceberg Integration Patterns

#aws #snowflake #dataengineering #apacheiceberg
5
Comments
7 min read
Backfilling Is Harder Than Scraping: Lessons From Rebuilding 6 Months of Missing Data
anna_6c67c00f5c3f53660978 profile
Anna

Backfilling Is Harder Than Scraping: Lessons From Rebuilding 6 Months of Missing Data

#webscraping #dataengineering #rapidproxy #architecture
Comments
3 min read
Designing Stable Integration Testing Architectures for Data-Driven Systems By QA Transformation & Integration Architect
chirag_patel_da672dcd5a8e profile

Designing Stable Integration Testing Architectures for Data-Driven Systems By QA Transformation & Integration Architect

#opensource #automation #azure #dataengineering
Comments
3 min read
How to Install Github Desktop for Data Analysis Projects
collins_muindi profile

How to Install Github Desktop for Data Analysis Projects

#beginners #python #datascience #dataengineering
Comments
2 min read
Your Ruby CSV Import Ran Successfully — Your Data May Still Be Wrong
tilo_sloboda profile

Your Ruby CSV Import Ran Successfully — Your Data May Still Be Wrong

#ruby #csv #rails #dataengineering
1
Comments
2 min read
🚀 Project Pravah: Powering High-Volume Real-Time Dashboards at Scale
optiwariindia profile

🚀 Project Pravah: Powering High-Volume Real-Time Dashboards at Scale

#showdev #analytics #dataengineering #performance
1
Comments 1
2 min read
Linux Fundamentals Every Beginner Data Engineer Should Know
naibei_caleb_01e47eb5148b profile

Linux Fundamentals Every Beginner Data Engineer Should Know

#dataengineering #linux #beginners
Comments
4 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.