DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
What Is Data Lineage? Learn How to Trace Your Data’s Journey

What Is Data Lineage? Learn How to Trace Your Data’s Journey

Comments
2 min read
Building a Data Career: The Skills That Truly Matter

Building a Data Career: The Skills That Truly Matter

10
Comments
5 min read
You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

2
Comments 1
8 min read
Unable to emit metadata to DataHub GMS with Airflow - a solution

Unable to emit metadata to DataHub GMS with Airflow - a solution

Comments
4 min read
Snowflake RBAC 101 – Episode 2: Role Hierarchies & Least Privilege

Snowflake RBAC 101 – Episode 2: Role Hierarchies & Least Privilege

Comments
1 min read
Lightweight ETL with AWS Glue Python Shell, DuckDB, and PyIceberg

Lightweight ETL with AWS Glue Python Shell, DuckDB, and PyIceberg

3
Comments 1
7 min read
PyIceberg on AWS Lambda: Comparing GlueCatalog and REST Catalog Access Methods

PyIceberg on AWS Lambda: Comparing GlueCatalog and REST Catalog Access Methods

2
Comments
3 min read
Big Data Fundamentals: data lake

Big Data Fundamentals: data lake

Comments
6 min read
Big Data Fundamentals: delta lake with python

Big Data Fundamentals: delta lake with python

Comments
6 min read
The Rise of Real-Time Data: Why Batch Might Be Fading

The Rise of Real-Time Data: Why Batch Might Be Fading

10
Comments
3 min read
📚 A Complete Guide to Data Science Courses: How to Choose, What to Learn, and Where to Begin

📚 A Complete Guide to Data Science Courses: How to Choose, What to Learn, and Where to Begin

Comments
5 min read
Engineering with SOLID, DRY, KISS, YAGNI and GRASP

Engineering with SOLID, DRY, KISS, YAGNI and GRASP

1
Comments
16 min read
Three Formats Walk into a Lakehouse: Iceberg, Delta and Hudi in a Local Setup You Can Run on Your Laptop

Three Formats Walk into a Lakehouse: Iceberg, Delta and Hudi in a Local Setup You Can Run on Your Laptop

7
Comments 4
16 min read
Which is Best for Real Time Dashboards: Airbyte, Fivetran, or Estuary

Which is Best for Real Time Dashboards: Airbyte, Fivetran, or Estuary

1
Comments
6 min read
Big Data Fundamentals: delta lake project

Big Data Fundamentals: delta lake project

Comments
6 min read
Big Data Fundamentals: delta lake example

Big Data Fundamentals: delta lake example

Comments
5 min read
Big Data Fundamentals: delta lake

Big Data Fundamentals: delta lake

Comments
6 min read
Big Data Fundamentals: data warehouse with python

Big Data Fundamentals: data warehouse with python

2
Comments
6 min read
Big Data Fundamentals: data warehouse tutorial

Big Data Fundamentals: data warehouse tutorial

2
Comments
6 min read
Big Data Fundamentals: data warehouse project

Big Data Fundamentals: data warehouse project

2
Comments
6 min read
Personal Picks: Data Product News (July 9, 2025)

Personal Picks: Data Product News (July 9, 2025)

Comments
6 min read
Key Concepts Every Data Engineer Should Master

Key Concepts Every Data Engineer Should Master

4
Comments
6 min read
Snowflake RBAC 101

Snowflake RBAC 101

Comments
1 min read
Big Data Fundamentals: hbase with python

Big Data Fundamentals: hbase with python

1
Comments
6 min read
Big Data Fundamentals: hbase tutorial

Big Data Fundamentals: hbase tutorial

1
Comments
6 min read
loading...