DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Data Science Simplified: Tips for Aspiring Data Scientists in 2025

Data Science Simplified: Tips for Aspiring Data Scientists in 2025

1
Comments
4 min read
2025 Guide to Architecting an Iceberg Lakehouse

2025 Guide to Architecting an Iceberg Lakehouse

5
Comments
14 min read
Data Engineer as a Real-Time Algo Trader – Turning Pipelines into Profit (or at Least Trying)!

Data Engineer as a Real-Time Algo Trader – Turning Pipelines into Profit (or at Least Trying)!

2
Comments
13 min read
One Off to One Data Platform: Design with Intent [Part 2]

One Off to One Data Platform: Design with Intent [Part 2]

1
Comments
5 min read
Case Study: Creating an ETL Data Pipeline using AWS Services - Real-World Problem

Case Study: Creating an ETL Data Pipeline using AWS Services - Real-World Problem

Comments
2 min read
Choosing the right, real-time, Postgres CDC platform

Choosing the right, real-time, Postgres CDC platform

Comments
8 min read
ChatGPT Launches Pro: What's it Mean for Data Professionals?

ChatGPT Launches Pro: What's it Mean for Data Professionals?

2
Comments
4 min read
Introduction to Apache Kafka

Introduction to Apache Kafka

3
Comments 1
3 min read
Mastering Workflow Automation with Apache Airflow for Data Engineering

Mastering Workflow Automation with Apache Airflow for Data Engineering

Comments
6 min read
Mastering Twitter Data Collection: A Comprehensive Guide to Efficient Scraping Solutions

Mastering Twitter Data Collection: A Comprehensive Guide to Efficient Scraping Solutions

Comments
3 min read
Seaborn Cheat Sheet

Seaborn Cheat Sheet

1
Comments
2 min read
Jupyter Notebooks in Docker

Jupyter Notebooks in Docker

10
Comments 1
3 min read
🚀 Beyond Data Ingestion: Advanced Strategies for Optimizing API Data Pipelines

🚀 Beyond Data Ingestion: Advanced Strategies for Optimizing API Data Pipelines

4
Comments 1
3 min read
SQL "SELECT INTO" vs "INSERT INTO SELECT" statements.

SQL "SELECT INTO" vs "INSERT INTO SELECT" statements.

Comments
1 min read
ACID Properties in Databases: What Happens Without Them?

ACID Properties in Databases: What Happens Without Them?

5
Comments
6 min read
🕵️ OSINT: link company acronyms to Standard Occupation Classification w. Open Source LLMs

🕵️ OSINT: link company acronyms to Standard Occupation Classification w. Open Source LLMs

1
Comments 9
6 min read
10 Future Apache Iceberg Developments to Look forward to in 2025

10 Future Apache Iceberg Developments to Look forward to in 2025

1
Comments
13 min read
Data Architecture Best Practices

Data Architecture Best Practices

1
Comments
6 min read
🚀 Unlock the Power of ORC File Format 📊

🚀 Unlock the Power of ORC File Format 📊

5
Comments
1 min read
Setting up memory for Flink - Configuration

Setting up memory for Flink - Configuration

Comments
3 min read
Designing robust and scalable relational databases: A series of best practices.

Designing robust and scalable relational databases: A series of best practices.

17
Comments 5
17 min read
From Data to Decisions: How Machine Learning Works in 2025

From Data to Decisions: How Machine Learning Works in 2025

3
Comments
3 min read
Why Data Security is Broken and How to Fix it?

Why Data Security is Broken and How to Fix it?

1
Comments
5 min read
Mastering Dynamic Allocation in Apache Spark: A Practical Guide with Real-World Insights

Mastering Dynamic Allocation in Apache Spark: A Practical Guide with Real-World Insights

Comments
3 min read
OLAP (Online Analytical Processing)

OLAP (Online Analytical Processing)

5
Comments
3 min read
loading...