DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
How to rotate data using Pivot & Unpivot operators

How to rotate data using Pivot & Unpivot operators

3
Comments 2
3 min read
Apply CDC From MySQL To Clickhouse on local environment

Apply CDC From MySQL To Clickhouse on local environment

7
Comments
6 min read
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

4
Comments
1 min read
Apache Flink vs Apache Spark: A detailed comparison for data processing

Apache Flink vs Apache Spark: A detailed comparison for data processing

15
Comments 1
5 min read
Abstract Configurations

Abstract Configurations

1
Comments
3 min read
Apache Flink episode 1: A comprehensive introduction

Apache Flink episode 1: A comprehensive introduction

2
Comments
6 min read
Data sources episode 2: AWS S3 to Postgres Data Sync using Singer

Data sources episode 2: AWS S3 to Postgres Data Sync using Singer

3
Comments
4 min read
Data sources episode 1: Common data sources in modern pipelines

Data sources episode 1: Common data sources in modern pipelines

1
Comments
6 min read
Handling NULL in the DBs

Handling NULL in the DBs

5
Comments 1
2 min read
Class to Airflow Custom Operator

Class to Airflow Custom Operator

Comments
3 min read
Scraper Function to Airflow DAG

Scraper Function to Airflow DAG

5
Comments 1
3 min read
From Class to Abstract Classes

From Class to Abstract Classes

1
Comments
3 min read
SQL 102:Intermediate SQL

SQL 102:Intermediate SQL

Comments
10 min read
From Functional to Class: a look at SOLID coding

From Functional to Class: a look at SOLID coding

1
Comments
3 min read
Hadoop Migration: How we pulled this off together

Hadoop Migration: How we pulled this off together

Comments
8 min read
Quick Detour on Unit Testing with PyTest

Quick Detour on Unit Testing with PyTest

1
Comments
3 min read
Unleashing the Magic of Job Schedulers: How to Tame Your Code and Save Your Sanity

Unleashing the Magic of Job Schedulers: How to Tame Your Code and Save Your Sanity

4
Comments
3 min read
Trigger Azure Data Factory Pipeline from Event Grid (Using Webhook Endpoint)

Trigger Azure Data Factory Pipeline from Event Grid (Using Webhook Endpoint)

7
Comments 4
4 min read
Mage Battlegrounds: Craft insights from real-time customer behavior analysis

Mage Battlegrounds: Craft insights from real-time customer behavior analysis

2
Comments
2 min read
Bootstrapped to Functional

Bootstrapped to Functional

1
Comments
3 min read
All about Structure Query Language (SQL)

All about Structure Query Language (SQL)

Comments
10 min read
AWS Cloud9 for Data Engineers

AWS Cloud9 for Data Engineers

1
Comments
5 min read
The Pyramid of Alerting

The Pyramid of Alerting

7
Comments 1
6 min read
Code optimization

Code optimization

Comments
2 min read
Batch Processing vs Stream Processing: Why Batch is dying and Streaming takes over

Batch Processing vs Stream Processing: Why Batch is dying and Streaming takes over

Comments
14 min read
loading...