DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Why Data SLAs Fail — and How to Enforce Them with a Unified Reliability Framework

Why Data SLAs Fail — and How to Enforce Them with a Unified Reliability Framework

Comments
2 min read
Unveiling the Power of Databases in the Realm of Big Data

Unveiling the Power of Databases in the Realm of Big Data

Comments
2 min read
Building a Medical-Grade Knowledge Graph: Mapping Drug Interactions with Neo4j and LlamaIndex 🩺💻

Building a Medical-Grade Knowledge Graph: Mapping Drug Interactions with Neo4j and LlamaIndex 🩺💻

Comments
3 min read
When an AI Suggests DataFrame.append: Missing Pandas Deprecations in Generated Code

When an AI Suggests DataFrame.append: Missing Pandas Deprecations in Generated Code

Comments 1
3 min read
Analysing Drivers of Digital Transformation in Corporate Innovation Capacity Using Amazon SageMaker Studio and Kaggle API

Analysing Drivers of Digital Transformation in Corporate Innovation Capacity Using Amazon SageMaker Studio and Kaggle API

Comments
2 min read
Glue Spark frequently used code snippets and configuration

Glue Spark frequently used code snippets and configuration

Comments
3 min read
Exploring Dynamic Return Types in PySpark pandas_udf

Exploring Dynamic Return Types in PySpark pandas_udf

Comments
2 min read
Day 30: From Zero to Production-Ready Spark Data Engineer

Day 30: From Zero to Production-Ready Spark Data Engineer

Comments
2 min read
OLTP y OLAP: Sistemas de Procesamiento de Datos Empresariales

OLTP y OLAP: Sistemas de Procesamiento de Datos Empresariales

Comments
5 min read
Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026

Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026

Comments
5 min read
JSON is Not Enough: The Engineering Headache of Flattening FHIR for Analytics

JSON is Not Enough: The Engineering Headache of Flattening FHIR for Analytics

Comments
4 min read
Unified Data Fabric: Serverless Spark on ROSA Integrating with AWS Glue Catalog

Unified Data Fabric: Serverless Spark on ROSA Integrating with AWS Glue Catalog

Comments
18 min read
Day 28: Spark Streaming Performance Tuning

Day 28: Spark Streaming Performance Tuning

Comments
1 min read
Day 27: Building Exactly-Once Streaming Pipelines with Spark & Delta Lake

Day 27: Building Exactly-Once Streaming Pipelines with Spark & Delta Lake

Comments
1 min read
Day 29: Building a Production-Grade Real-Time ETL Pipeline with Spark & Delta

Day 29: Building a Production-Grade Real-Time ETL Pipeline with Spark & Delta

Comments
1 min read
Production AI: Monitoring, Cost Optimization, and Operations

Production AI: Monitoring, Cost Optimization, and Operations

Comments
9 min read
Building a Realistic Banking Dummy Data Generator with Bad-Data Simulation

Building a Realistic Banking Dummy Data Generator with Bad-Data Simulation

Comments
1 min read
Data Engineering Trends You Can’t Ignore in 2026

Data Engineering Trends You Can’t Ignore in 2026

Comments
5 min read
Day 26: Spark Streaming Joins

Day 26: Spark Streaming Joins

Comments
1 min read
DataOps 101: What It Is and Why Enterprises Can’t Ignore It in 2026

DataOps 101: What It Is and Why Enterprises Can’t Ignore It in 2026

Comments
2 min read
Day 25: Streaming Aggregations in Spark

Day 25: Streaming Aggregations in Spark

Comments
1 min read
What Is Data Fabric Architecture? A Beginner’s Guide (Explained Simply)

What Is Data Fabric Architecture? A Beginner’s Guide (Explained Simply)

Comments
2 min read
Data Processing Does Not Belong in the Message Broker

Data Processing Does Not Belong in the Message Broker

Comments
3 min read
Can you describe a complex data architecture you’ve designed or implemented in the past?

Can you describe a complex data architecture you’ve designed or implemented in the past?

Comments
1 min read
Apache SeaTunnel Community Year-End Review 2025

Apache SeaTunnel Community Year-End Review 2025

1
Comments
7 min read
loading...