DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
The "Shift-Left" Imperative: Implementing Data Contracts in CI/CD Pipeline
nabindebnath profile

The "Shift-Left" Imperative: Implementing Data Contracts in CI/CD Pipeline

#dataengineering #cicd #devops #shiftleft
1
Comments
4 min read
Break Through Data Silos: Practices of Multi-cloud Observability Integration Based on Object Storage Service (OSS)
observabilityguy profile

Break Through Data Silos: Practices of Multi-cloud Observability Integration Based on Object Storage Service (OSS)

#dataengineering #cloud #architecture #monitoring
Comments
12 min read
Building a Universal Lakehouse Catalog: Beyond Iceberg Tables
alexmercedcoder profile

Building a Universal Lakehouse Catalog: Beyond Iceberg Tables

#bigdata #dataengineering #architecture #database
Comments
10 min read
Tutorial: Intro to Apache Iceberg with Apache Polaris and Apache Spark
alexmercedcoder profile

Tutorial: Intro to Apache Iceberg with Apache Polaris and Apache Spark

#bigdata #dataengineering #opensource #tutorial
2
Comments 4
20 min read
🎓 Building a Smart LMS Assistant: RAG System with Pinecone for Multi-Source Learning Data
fagundesv profile

🎓 Building a Smart LMS Assistant: RAG System with Pinecone for Multi-Source Learning Data

#vectordatabase #ai #dataengineering
Comments
3 min read
Comprehensive Guide: kwargs vs XCom in Python & Airflow
lagat_josiah_f024a2c855bc profile

Comprehensive Guide: kwargs vs XCom in Python & Airflow

#dataengineering #python #tutorial
Comments
4 min read
Precise Data Extraction: Pattern-Based Partitioning for Structured Extraction
drguthals profile

Precise Data Extraction: Pattern-Based Partitioning for Structured Extraction

#automation #dataengineering #llm
1
Comments
3 min read
Apache Gravitino 1.0.0 — From Metadata Management to Contextual Engineering
alex_yan_163de34c186edd87 profile

Apache Gravitino 1.0.0 — From Metadata Management to Contextual Engineering

#dataengineering #opensource #cloud #architecture
1
Comments
7 min read
Chinese DBA's Story: Hu Zhonghao - The Journey of Becoming a DBA for Domestic Distributed Databases
tripcat profile

Chinese DBA's Story: Hu Zhonghao - The Journey of Becoming a DBA for Domestic Distributed Databases

#database #oracle #dataengineering
Comments 1
7 min read
Apache Kafka in Data engineering
eden_wetende profile

Apache Kafka in Data engineering

#dataengineering #kafka
6
Comments 1
1 min read
🧭System Design Roadmap for Data Engineers
sajjadrahman56 profile

🧭System Design Roadmap for Data Engineers

#database #dataengineering #systemdesign #beginners
5
Comments
3 min read
Orchestrating and Observing Data Pipelines with Airflow, PostgreSQL, and Polar
ctrix profile

Orchestrating and Observing Data Pipelines with Airflow, PostgreSQL, and Polar

#postgres #dataengineering #docker #devops
1
Comments
3 min read
đź’Ą Polars vs. Pandas: Why Your Next ETL Pipeline Should Run on Rust (Part 1/5)
narayan_f8f2c91c99dfd33e6 profile

đź’Ą Polars vs. Pandas: Why Your Next ETL Pipeline Should Run on Rust (Part 1/5)

#dataengineering #rust #python #performance
1
Comments
2 min read
Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK
andreparis profile

Building a Production-Ready Data Lake: PostgreSQL to S3 with AWS DMS, Glue, and Athena using CDK

#aws #dataengineering #typescript #postgres
Comments
8 min read
(â…ˇ) A Complete Guide to Core Data Warehouse Design Standards: From Layers, Types to Lifecycle
seatunnel profile

(â…ˇ) A Complete Guide to Core Data Warehouse Design Standards: From Layers, Types to Lifecycle

#datascience #bigdata #datawarehouse #dataengineering
Comments
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.