# dataengineering

6 min read

Sridhar CR

Dec 13

How to Migrate Massive Data in Record Time—Without a Single Minute of Downtime 🕑

#etl #spark #database #dataengineering

4 min read

SNEHASISH DUTTA

Dec 9

Data Engineer as a Real-Time Algo Trader – Turning Pipelines into Profit (or at Least Trying)!

#sql #python #dataengineering #eventdriven

13 min read

ksanaka

Dec 20

Easily Integrate Databend Test Environment with Testcontainers

#dataengineering #rust #testcontainers #opensource

4

4 min read

Eric Goldman for Sequin

Dec 6

Choosing the right, real-time, Postgres CDC platform

#postgres #database #eventdriven #dataengineering

8 min read

pawan deore

Dec 1

Optimizing Large-Scale Data Processing in Python: A Guide to Parallelizing CSV Operations

#webdev #python #csv #dataengineering

3 min read

Arum Puri

Nov 30

Seaborn Cheat Sheet

#seaborn #datascience #cheatsheet #dataengineering

2 min read

HASNAA ESSOUNAINI

Nov 28

Should I add Data Science or Analytics to my skills?

#webdev #datascience #dataengineering #analytics

1 min read

Mikalai

Nov 28

Innowise is open for internships for Data Engineers and Data Analytics

#beginners #career #dataengineering

1 min read

Tim Huang

Dec 18

Query 1B Rows in PostgreSQL >25x Faster with Squirrels!

#postgres #dataengineering #analytics #bigdata

8

5 min read

Alex Merced

Nov 25

10 Future Apache Iceberg Developments to Look forward to in 2025

#database #datascience #dataengineering #dataanalytics

13 min read

Pablo

Dec 15

📊 AI Dashboard Builder: Create Insightful Dashboards just Droppping your Data

#dataengineering #datascience #llm #python

2 min read

kination

Nov 22

Setting up memory for Flink - Configuration

#dataengineering #flink #memory #tuning

3 min read

Vaibhav Shirpurkar

Nov 17

Mastering Dynamic Allocation in Apache Spark: A Practical Guide with Real-World Insights

#pyspark #spark #dataengineering #data

3 min read

Hana Sato

Nov 13

Talend vs. Apache Kafka: Which Data Tool Drives Better Business Insights?

#datascience #devops #dataengineering #datastructures

6 min read

Omar Urbano | LightningChart

Nov 12

LightningChart Python 1.0

#python #datascience #dataengineering #datavisualization

1 min read

Hiswill Thompson

Dec 14

Introduction to Data lakes: The future of big data storage

#bigdata #dataengineering

10

2 min read

Yann Barrault for Onepoint

Dec 14

Explorer l'API de 360Learning : de l'agilité de Power Query à la robustesse de la Modern Data Stack

#adventoftech2024 #dataengineering #airbyte #dbt

6

12 min read

Muhammad Mahdi Ramadhan

Nov 9

Data Pipeline Filters 101: Choosing Between Static and Dynamic Approaches

#dataengineering #etl #dataanalyst #database

1 min read

Danica Fine

Dec 11

The Apache Iceberg™ Small File Problem

#bigdata #apacheiceberg #datalakehouse #dataengineering

3 min read

BuzzGK

Nov 7

Ensuring Data Quality: Best Practices and Automation

#data #dataengineering

6 min read

Vikas76

Nov 10

Data Science Simplified: Tips for Aspiring Data Scientists in 2025

#datascience #dataengineering #data #roadmap

4 min read

Alex Merced

Dec 9

2025 Guide to Architecting an Iceberg Lakehouse

#apacheiceberg #database #dataengineering #datascience

13

14 min read

Alex Merced

Nov 5

Dremio, Apache Iceberg and their role in AI-Ready Data

#database #dataengineering #datascience #dataanalytics

7 min read

Alex Merced

Nov 4

Leveraging Python's Pattern Matching and Comprehensions for Data Analytics

#python #datascience #dataengineering #database

12 min read

Lulu Cheng for jarrid.xyz

Dec 8

One Off to One Data Platform: Design with Intent [Part 2]

#data #architecture #dataengineering #startup

9

5 min read

M. Abdullah Bin Aftab

Dec 6

Case Study: Creating an ETL Data Pipeline using AWS Services - Real-World Problem

#aws #dataengineering #data #cloudcomputing

2 min read

Puneet Verma

Nov 16

Understanding Star Schema vs. Snowflake Schema

#dataengineering #datascience #datamodeling #bigdata

1 min read

Anthony Clemons

Dec 5

ChatGPT Launches Pro: What's it Mean for Data Professionals?

#openai #chatgpt #datascience #dataengineering

4 min read

Hiswill Thompson

Dec 5

Introduction to Apache Kafka

#datapipeline #dataengineering #realtimedat

4

3 min read

Aditya Pratap Bhuyan

Oct 31

Mastering Workflow Automation with Apache Airflow for Data Engineering

#dataengineering #apacheairflow

6 min read

kaito

Dec 3

Mastering Twitter Data Collection: A Comprehensive Guide to Efficient Scraping Solutions

#twitter #datascience #dataengineering #development

3 min read

Hassan Aftab

Nov 29

Jupyter Notebooks in Docker

#datascience #docker #dataengineering #programming

4

3 min read

SANKET PATIL

Nov 29

🚀 Beyond Data Ingestion: Advanced Strategies for Optimizing API Data Pipelines

#dataengineering #azuredatafactory #apipagination #devops

3

3 min read

Danwycliff Ndwiga

Oct 30

SQL "SELECT INTO" vs "INSERT INTO SELECT" statements.

#sql #database #data #dataengineering

1 min read

Meqdad Darwish

Nov 27

ACID Properties in Databases: What Happens Without Them?

#database #data #dataengineering #sql

6 min read

adriens

Nov 25

🕵️ OSINT: link company acronyms to Standard Occupation Classification w. Open Source LLMs

#ai #datascience #database #dataengineering

8

6 min read

DQOps

Nov 23

Data Architecture Best Practices

#data #dataengineering #dataquality #datascience

6 min read

Lusanda Ndlovu

Oct 20

My Journey into Data AI and Machine Learning

#softwaredevelopment #ai #machinelearning #dataengineering

1 min read

Pratik Barjatiya

Nov 22

🚀 Unlock the Power of ORC File Format 📊

#dataengineering #bigdata #datascience #data

1 min read

Akhilesh Pratap Shahi

Nov 10

The Ultimate Data Engineering Roadmap: From Beginner to Pro

#dataengineering #datascience #computerscience #machinelearning

6

8 min read

Pedro H Goncalves

Nov 19

Designing robust and scalable relational databases: A series of best practices.

#database #advanced #optimization #dataengineering

10

17 min read

Vikas76

Nov 20

From Data to Decisions: How Machine Learning Works in 2025

#machinelearning #datascience #data #dataengineering

3 min read

Lulu Cheng for jarrid.xyz

Oct 15

Why Data Security is Broken and How to Fix it?

#security #automation #devops #dataengineering

5 min read

luminousmen

Oct 15

From ETL and ELT to Reverse ETL

#dataengineering #bigdata #data

4 min read

Rodolfo Mendivil

Oct 18

Mastering Informatica Intelligent Cloud Services (IICS) for Cloud Data Integration

#iics #data #etl #dataengineering

3 min read

Pranav Bakare

Nov 16

OLAP (Online Analytical Processing)

#datascience #database #data #dataengineering

3 min read

1:42:26

Daniel Davis for TrustGraph

Nov 15

The Future of Agentic Systems Podcast

#ai #aiops #opensource #dataengineering

6

1 min read

Norton Augusto Herrero dos Santos

Oct 12

What is Data Engineering?

#dataengineering #datascience

1 min read

Alex Merced

Nov 15

Deep Dive into Dremio's File-based Auto Ingestion into Apache Iceberg Tables

#database #dataengineering #datascience

13 min read

Lulu Cheng for jarrid.xyz

Nov 14

One Off to One Data Platform: The Unscalable Data Platform [Part 1]

#data #architecture #devops #dataengineering

3 min read

Hana Sato

Nov 13

What are the major advantages of a cloud warehouse solution over an on-premises data warehouse solution?

#data #datascience #dataengineering

1

5 min read

Hana Sato

Nov 13

Databricks vs. Hadoop: Which Platform is Best for Predictive Analytics?

#dataengineering #analytics #tutorial #ai