DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
How to Use Apache Airflow to Get 1000+ Files From a Public Dataset
cemkeskin84 profile

How to Use Apache Airflow to Get 1000+ Files From a Public Dataset

#dezoomcamp #dataengineering #airflow #publicdatasets
8
Comments
10 min read
ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)
rubnsbarbosa profile

ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)

#etl #dataengineering #python #azure
10
Comments
5 min read
What is Azure Synapse Analytics?
integerman profile

What is Azure Synapse Analytics?

#data #dataengineering #azure #cloud
4
Comments
7 min read
Design concept of a best opensource project about big data and data lakehouse
dmetasoul profile

Design concept of a best opensource project about big data and data lakehouse

#opensource #dataengineering #bigdata #datascience
9
Comments
9 min read
Details of 4 best opensource projects about big data you should try out（Ⅰ）
dmetasoul profile

Details of 4 best opensource projects about big data you should try out（Ⅰ）

#opensource #dataengineering #bigdata #spark
8
Comments
5 min read
Debezium Change Data Capture without Kafka Connect
tchiotludo profile

Debezium Change Data Capture without Kafka Connect

#database #dataengineering #opensource #etl
10
Comments 1
8 min read
Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment
dmetasoul profile

Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment

#opensource #dataengineering #bigdata #spark
8
Comments
5 min read
4 best opensource projects about big data you should try out
dmetasoul profile

4 best opensource projects about big data you should try out

#opensource #dataengineering #bigdata #spark
16
Comments 3
3 min read
A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions
dmetasoul profile

A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions

#dataengineering #opensource #bigdata #programming
8
Comments
2 min read
Modern data warehouse patterns: ELT with Snowflake variants
biellls profile

Modern data warehouse patterns: ELT with Snowflake variants

#snowflake #dataengineering #etl #elt
9
Comments
6 min read
Standing on the shoulders of giants. Part one: Airflow
biellls profile

Standing on the shoulders of giants. Part one: Airflow

#airflow #dataengineering #python #data
7
Comments
5 min read
Data Engineering in Julia
logankilpatrick profile

Data Engineering in Julia

#machinelearning #julia #dataengineering #julialang
4
Comments 1
1 min read
Extract csv data and load it to PostgreSQL using Meltano ELT
zompro profile

Extract csv data and load it to PostgreSQL using Meltano ELT

#python #meltano #dataengineering #elt
12
Comments
6 min read
Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster
tkeyo profile

Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster

#aws #dagster #dataengineering #python
5
Comments 6
10 min read
Implementing Graceful Shutdown in Go
teamrudderstack profile

Implementing Graceful Shutdown in Go

#go #dataengineering #sre #database
15
Comments 5
14 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.