DEV Community

# etl

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Using AWS Glue Studio for your ETL Jobs

Using AWS Glue Studio for your ETL Jobs

6
Comments
7 min read
Data architecture models

Data architecture models

3
Comments
6 min read
Apache Airflow. How to make the complex workflow as an easy job

Apache Airflow. How to make the complex workflow as an easy job

9
Comments 1
7 min read
Using Athena Views As A Source In Glue

Using Athena Views As A Source In Glue

16
Comments 3
4 min read
A simple serverless architecture on AWS ecosystem for data ETL and visualization

A simple serverless architecture on AWS ecosystem for data ETL and visualization

5
Comments
1 min read
Kestra, infinitely scalable open source orchestration and scheduling platform.

Kestra, infinitely scalable open source orchestration and scheduling platform.

4
Comments
6 min read
Modern data warehouse patterns: ELT with Snowflake variants

Modern data warehouse patterns: ELT with Snowflake variants

9
Comments
6 min read
How to Migrate from Segment to RudderStack

How to Migrate from Segment to RudderStack

6
Comments
8 min read
How To Build An ETL Using Python, Docker, PostgreSQL And Airflow

How To Build An ETL Using Python, Docker, PostgreSQL And Airflow

42
Comments
26 min read
Why It’s Hard for Engineering to Support Marketing

Why It’s Hard for Engineering to Support Marketing

2
Comments
3 min read
Part 2: The Evolution of Data Pipeline Architecture

Part 2: The Evolution of Data Pipeline Architecture

4
Comments
6 min read
What Is A Customer Data Pipeline?

What Is A Customer Data Pipeline?

3
Comments 1
4 min read
How To Event Stream From Your Gatsby Website Using Open Source RudderStack

How To Event Stream From Your Gatsby Website Using Open Source RudderStack

5
Comments
8 min read
Cloud data warehouse architectures

Cloud data warehouse architectures

4
Comments
1 min read
Data warehouse explained

Data warehouse explained

5
Comments
2 min read
Part 1: The Evolution of Data Pipeline Architecture

Part 1: The Evolution of Data Pipeline Architecture

1
Comments
6 min read
RudderStack + Blendo: Better Together

RudderStack + Blendo: Better Together

2
Comments
7 min read
Starting small Airbyte on GCP

Starting small Airbyte on GCP

9
Comments
5 min read
RudderStack Product News Vol. #016 - Warehouse Actions Mirror Sync Mode

RudderStack Product News Vol. #016 - Warehouse Actions Mirror Sync Mode

3
Comments
1 min read
Data Engineering:Extract, Transform,and Load Using Talend Open Studio.

Data Engineering:Extract, Transform,and Load Using Talend Open Studio.

20
Comments 1
3 min read
Find The Best Way To Load Data In A Data Warehouse

Find The Best Way To Load Data In A Data Warehouse

2
Comments
4 min read
Extract, Transform and Load with React & Rails

Extract, Transform and Load with React & Rails

16
Comments
4 min read
SQL SERVER REMOTE CONFIGURATIONS ON LINUX

SQL SERVER REMOTE CONFIGURATIONS ON LINUX

5
Comments
2 min read
JETL - J Extract Transform and Load

JETL - J Extract Transform and Load

5
Comments 1
9 min read
The Data Trinity

The Data Trinity

5
Comments
4 min read
Running SSIS Packages with Python

Running SSIS Packages with Python

6
Comments
3 min read
AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData

AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData

6
Comments
3 min read
Primeiros passos com o Apache Airflow

Primeiros passos com o Apache Airflow

21
Comments
5 min read
ETL (Extract, Transform, Load). Best Practices ETL Process And Lifehacks

ETL (Extract, Transform, Load). Best Practices ETL Process And Lifehacks

3
Comments
11 min read
Fixing a 40-year-old Software Bug

Fixing a 40-year-old Software Bug

27
Comments 4
6 min read
Insert, update, and delete from a database to Salesforce

Insert, update, and delete from a database to Salesforce

5
Comments
9 min read
ETL com Apache Airflow, Web Scraping, AWS S3, Apache Spark e Redshift | Parte 1

ETL com Apache Airflow, Web Scraping, AWS S3, Apache Spark e Redshift | Parte 1

20
Comments 1
7 min read
First Look: AWS Glue DataBrew

First Look: AWS Glue DataBrew

10
Comments
7 min read
ETL & Enterprise Level Practices

ETL & Enterprise Level Practices

5
Comments
7 min read
Automate PDF Table Extraction using Tabula and Azure Functions

Automate PDF Table Extraction using Tabula and Azure Functions

2
Comments 1
5 min read
Completing the #CloudGuruChallenge – Event-Driven Python on AWS from ACloudGuru

Completing the #CloudGuruChallenge – Event-Driven Python on AWS from ACloudGuru

2
Comments
3 min read
Cut data warehouse costs with run caching

Cut data warehouse costs with run caching

5
Comments
3 min read
Dagster with User Code Deployments (gRPC)

Dagster with User Code Deployments (gRPC)

18
Comments 2
6 min read
#CloudGuruChallenge – Event-Driven Python on AWS

#CloudGuruChallenge – Event-Driven Python on AWS

2
Comments
3 min read
#CloudGuruChallenge – Event-Driven Python on AWS - Completed!

#CloudGuruChallenge – Event-Driven Python on AWS - Completed!

6
Comments
4 min read
Luigi for data pipelines - things I like.

Luigi for data pipelines - things I like.

6
Comments
3 min read
Streaming data into Kafka S01/E03 - Loading JSON file

Streaming data into Kafka S01/E03 - Loading JSON file

9
Comments
9 min read
4 lessons before starting your cloud migration

4 lessons before starting your cloud migration

2
Comments
11 min read
How To Run Airflow on Windows (with Docker)

How To Run Airflow on Windows (with Docker)

20
Comments 3
8 min read
Manipulating Data with PHP: performing ETL operations

Manipulating Data with PHP: performing ETL operations

5
Comments
3 min read
CI/CD for ETL/ELT pipelines

CI/CD for ETL/ELT pipelines

18
Comments
3 min read
10 Key skills, to help you become a data engineer

10 Key skills, to help you become a data engineer

9
Comments
3 min read
Neo4j – SSIS – Connection Manager Love

Neo4j – SSIS – Connection Manager Love

4
Comments
1 min read
Dynamic ETL from RDS to Redshift using AWS Glue

Dynamic ETL from RDS to Redshift using AWS Glue

6
Comments
3 min read
Azure: Passing status messages and results back from Databricks to ADF

Azure: Passing status messages and results back from Databricks to ADF

7
Comments 2
2 min read
Kafka Connect: How it let us down?

Kafka Connect: How it let us down?

5
Comments 1
5 min read
Neo4j & NiFi – Getting NiFi Running

Neo4j & NiFi – Getting NiFi Running

2
Comments
4 min read
A microservice making electorate info more accessible

A microservice making electorate info more accessible

8
Comments
1 min read
Why Postman Data Engineering chose Apache Spark for ETL (Extract-Transform-Load)

Why Postman Data Engineering chose Apache Spark for ETL (Extract-Transform-Load)

28
Comments 1
6 min read
Data Ingestion with Azure Event Hubs using Python

Data Ingestion with Azure Event Hubs using Python

8
Comments
2 min read
Reinventing SSIS scripting with JavaScript - COZYROC

Reinventing SSIS scripting with JavaScript - COZYROC

8
Comments
1 min read
ON the evolution of Data Engineering

ON the evolution of Data Engineering

15
Comments
4 min read
Manage Data Pipelines with Apache Airflow

Manage Data Pipelines with Apache Airflow

76
Comments
13 min read
Reducing the Need for ETL with MongoDB Charts

Reducing the Need for ETL with MongoDB Charts

5
Comments
8 min read
Parallelising ETL workflows with the Jongleur gem

Parallelising ETL workflows with the Jongleur gem

5
Comments
7 min read
loading...