DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
From Postgres to Iceberg
misachi profile

From Postgres to Iceberg

#database #postgres #dataengineering
1
Comments
11 min read
isql
query_filter_591122b53770 profile

isql

#automation #dataengineering #sql
Comments
1 min read
The data lakehouse evolution
apachedoris profile

The data lakehouse evolution

#bigdata #lakehouse #dataengineering #olap
Comments
11 min read
Digital Vaults: Unlocking the Future of Data Archiving with Cutting-Edge Databases
vjnvisakh profile

Digital Vaults: Unlocking the Future of Data Archiving with Cutting-Edge Databases

#dataengineering #security #architecture #database
Comments
2 min read
OSPP Project Outcome: Supporting Flink Engine CDC Source Schema Evolution
seatunnel profile

OSPP Project Outcome: Supporting Flink Engine CDC Source Schema Evolution

#database #dataengineering #opensource
Comments
6 min read
Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS
datapenguin profile

Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS

#aws #dataengineering #lambda #sqs
2
Comments
6 min read
Interesting links - October 2025
rmoff profile

Interesting links - October 2025

#data #dataengineering #kafka #flink
Comments
15 min read
Data Cataloguing in AWS
devoteng1 profile

Data Cataloguing in AWS

#aws #dataengineering #tutorial #architecture
Comments
5 min read
From Dashboards to Decisions: Building Scalable Self-Service BI for Real Impact
laxmivanam profile

From Dashboards to Decisions: Building Scalable Self-Service BI for Real Impact

#dataengineering #analytics #businessintelligence #datamodeling
Comments
2 min read
Real-World Strategies for Scaling AI in Large Organizations
krunalbhimani profile

Real-World Strategies for Scaling AI in Large Organizations

#ai #machinelearning #mlops #dataengineering
Comments
3 min read
Clean Code in ETL:How Python, Go, and SQL Each Teach You to Think Differently
mohhddhassan profile

Clean Code in ETL:How Python, Go, and SQL Each Teach You to Think Differently

#dataengineering #python #go #sql
2
Comments
3 min read
Medallion Architecture On AWS
devoteng1 profile

Medallion Architecture On AWS

#data #aws #dataengineering #architecture
Comments
4 min read
🧠Understanding 6 Common Data Formats in Cloud Data Analytics
cibin_s profile

🧠Understanding 6 Common Data Formats in Cloud Data Analytics

#analytics #data #dataengineering #cloudcomputing
1
Comments
3 min read
Introducing ReelTrust: What if data engineering could solve our AI deepfakes problem?
aaronsteers profile

Introducing ReelTrust: What if data engineering could solve our AI deepfakes problem?

#ai #dataengineering #opensource
Comments
5 min read
Who Needs Real-Time Streaming? Use Cases & Architecture Across Industries
vijaya_bhaskarv_ba95adf9 profile

Who Needs Real-Time Streaming? Use Cases & Architecture Across Industries

#systemdesign #dataengineering #performance #architecture
Comments
8 min read
From “I want automation” to “It runs”: 15 decisions for lead enrichment that actually execute
maybe_ai profile

From “I want automation” to “It runs”: 15 decisions for lead enrichment that actually execute

#automation #sales #dataengineering #ai
Comments
3 min read
Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing
yks profile

Daft vs Ray Data: A Comprehensive Comparison for Multimodal Data Processing

#dataengineering #aiops #distributedsystems #machinelearning
10
Comments
6 min read
End-to-End Data Workflow: Kestra, Redshift, and dbt Integration
fredmunjogu profile

End-to-End Data Workflow: Kestra, Redshift, and dbt Integration

#dataengineering #automation #tutorial #aws
4
Comments
9 min read
What Does a Data Engineer Do? Understanding the Role and Career Path
directdata_education_83e6 profile

What Does a Data Engineer Do? Understanding the Role and Career Path

#beginners #career #dataengineering
1
Comments
6 min read
Auto-Detecting CSV Schemas for Lightning-Fast ClickHouse Ingestion with Parquet
darkedges profile

Auto-Detecting CSV Schemas for Lightning-Fast ClickHouse Ingestion with Parquet

#node #dataengineering #database #automation
7
Comments
5 min read
Smart Invoice Analyzer — How I Automated Invoice Processing & Predicted Sales Using Machine Learning
sanjaydot profile

Smart Invoice Analyzer — How I Automated Invoice Processing & Predicted Sales Using Machine Learning

#python #businessintelligence #machinelearning #dataengineering
3
Comments
2 min read
How I Streamed Live Binance L2 Order Book Data on AWS for ~$10/Month
goodiegoody profile

How I Streamed Live Binance L2 Order Book Data on AWS for ~$10/Month

#aws #fintech #infrastructure #dataengineering
5
Comments
14 min read
Stop Waiting for the Cloud: Building a Hybrid SQL+Python Data Pipeline Locally with DuckDB
markolekic profile

Stop Waiting for the Cloud: Building a Hybrid SQL+Python Data Pipeline Locally with DuckDB

#python #sql #dataengineering #tooling
Comments
5 min read
Decommissioning the Dinosaur: A 4-Phase Playbook for Migrating Your Legacy Data Warehouse to Databricks
shubhojeet2001 profile

Decommissioning the Dinosaur: A 4-Phase Playbook for Migrating Your Legacy Data Warehouse to Databricks

#datascience #databricks #devops #dataengineering
Comments
4 min read
The Data Analytics Lifecycle
hammoudi_wissem_2a55b8cc2 profile

The Data Analytics Lifecycle

#dataengineering #analytics
Comments
3 min read
loading...