DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Introduction to data engineering

Introduction to data engineering

5
Comments
4 min read
Create Jira Ticket on Prefect Task Failure

Create Jira Ticket on Prefect Task Failure

1
Comments
2 min read
Hash Personal Identifiable Information (PII) in your ELT pipelines

Hash Personal Identifiable Information (PII) in your ELT pipelines

3
Comments
3 min read
Difference Between Data Engineer and Data Scientist?

Difference Between Data Engineer and Data Scientist?

7
Comments
3 min read
Learning Workflow Schedulers (Oozie)

Learning Workflow Schedulers (Oozie)

2
Comments
5 min read
Solving AttributeError: 'float' object has no attribute 'rint'

Solving AttributeError: 'float' object has no attribute 'rint'

5
Comments
2 min read
[Spark-k8s] — Getting started # Part 1

[Spark-k8s] — Getting started # Part 1

3
Comments
4 min read
Websites to find Dataset for your Data Engineering projects.

Websites to find Dataset for your Data Engineering projects.

5
Comments
1 min read
Data engineers must-see: The future trend of big data cloud services

Data engineers must-see: The future trend of big data cloud services

8
Comments 1
8 min read
Data Engineering Projects for Beginners

Data Engineering Projects for Beginners

24
Comments 2
2 min read
Data Pipelines with Apache Airflow - Book Review

Data Pipelines with Apache Airflow - Book Review

8
Comments
2 min read
ETL vs Interactive Queries: The Case for Both

ETL vs Interactive Queries: The Case for Both

6
Comments 1
8 min read
Data Engineering - Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow

Data Engineering - Creating a Streaming Data Pipeline for a Real-Time Dashboard with Dataflow

10
Comments
4 min read
Parsing logs from multiple data sources with Ahana and Cube

Parsing logs from multiple data sources with Ahana and Cube

14
Comments
24 min read
Solved a practical business problem when using Hudi: LakeSoul supports null field non-override semanticssemantics

Solved a practical business problem when using Hudi: LakeSoul supports null field non-override semanticssemantics

7
Comments
3 min read
What is the Lakehouse, the latest Direction of Big Data Architecture?

What is the Lakehouse, the latest Direction of Big Data Architecture?

9
Comments
10 min read
Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL

Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL

7
Comments
6 min read
Using dbt for Transformation Tasks on BigQuery

Using dbt for Transformation Tasks on BigQuery

10
Comments 1
4 min read
Docker and Kubernetes

Docker and Kubernetes

6
Comments
3 min read
How to Use Apache Airflow to Get 1000+ Files From a Public Dataset

How to Use Apache Airflow to Get 1000+ Files From a Public Dataset

8
Comments
10 min read
ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)

ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)

10
Comments
5 min read
What is Azure Synapse Analytics?

What is Azure Synapse Analytics?

4
Comments
7 min read
Design concept of a best opensource project about big data and data lakehouse

Design concept of a best opensource project about big data and data lakehouse

9
Comments
9 min read
When To Build vs. Buy Data Pipelines

When To Build vs. Buy Data Pipelines

3
Comments
6 min read
Details of 4 best opensource projects about big data you should try out(Ⅰ)

Details of 4 best opensource projects about big data you should try out(Ⅰ)

8
Comments
5 min read
loading...