DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Understanding Data Pipelines: The Backbone of Modern Data Systems

Understanding Data Pipelines: The Backbone of Modern Data Systems

1
Comments
3 min read
🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel

🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel

1
Comments 2
4 min read
Stock Data Extraction Using Apache Kafka

Stock Data Extraction Using Apache Kafka

6
Comments
4 min read
Time of YAML/JSON for data engineer

Time of YAML/JSON for data engineer

1
Comments
2 min read
Distributed Model Serving Patterns

Distributed Model Serving Patterns

1
Comments
4 min read
Building Automated Data Reports from Supabase with GitHub Actions and R Markdown

Building Automated Data Reports from Supabase with GitHub Actions and R Markdown

1
Comments
12 min read
"Our GPUs Are Melting": Building a Real-Time Streaming System with Go and Redpanda Ghibli Style

"Our GPUs Are Melting": Building a Real-Time Streaming System with Go and Redpanda Ghibli Style

1
Comments
10 min read
Thinking about becoming a Data Engineer?

Thinking about becoming a Data Engineer?

Comments
2 min read
🔬Public docker images Trivy scans as duckdb datas on Kaggle

🔬Public docker images Trivy scans as duckdb datas on Kaggle

Comments 5
1 min read
Apache SeaTunnel 2.3.8 JDBC Connector Development Guide

Apache SeaTunnel 2.3.8 JDBC Connector Development Guide

Comments
12 min read
Slash your cost by 90% with Apache Doris Compute-Storage Decoupled Mode

Slash your cost by 90% with Apache Doris Compute-Storage Decoupled Mode

Comments
9 min read
🐼 Pandas Too Slow? Try These Fast Python Libraries for Data Analysis

🐼 Pandas Too Slow? Try These Fast Python Libraries for Data Analysis

Comments
1 min read
Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra

Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra

Comments
1 min read
How to Optimize SQL Queries for Speed and Efficiency

How to Optimize SQL Queries for Speed and Efficiency

3
Comments 2
5 min read
Building an Automated Bitcoin Price ETL Pipeline with Airflow and PostgreSQL

Building an Automated Bitcoin Price ETL Pipeline with Airflow and PostgreSQL

3
Comments
3 min read
Apache Airflow for Data Engineering: Best Practices and Real-World Examples

Apache Airflow for Data Engineering: Best Practices and Real-World Examples

6
Comments 1
7 min read
Set up Graph Databases in Large-Scale Applications for Complex Data Management

Set up Graph Databases in Large-Scale Applications for Complex Data Management

Comments
3 min read
Implementing MLOps within Data Engineering Workflows for Efficient Machine Learning Model Deployment

Implementing MLOps within Data Engineering Workflows for Efficient Machine Learning Model Deployment

Comments
3 min read
Stop flattening your JSON just to export it to Excel

Stop flattening your JSON just to export it to Excel

Comments
2 min read
Ghost models and spooky manifests in dbt

Ghost models and spooky manifests in dbt

Comments
4 min read
Implementing Advanced Data Governance in Hybrid and Multi-Cloud Environments

Implementing Advanced Data Governance in Hybrid and Multi-Cloud Environments

Comments
4 min read
Azure Data Factory

Azure Data Factory

7
Comments
1 min read
SQLite Database GUI Tool | Design and Visualize with DbSchema

SQLite Database GUI Tool | Design and Visualize with DbSchema

9
Comments 1
4 min read
Amazon Redshift Data Warehousing (Sample Project)

Amazon Redshift Data Warehousing (Sample Project)

3
Comments
2 min read
Pair and Transpose Adjacent Records within the Group - From SQL to SPL #13

Pair and Transpose Adjacent Records within the Group - From SQL to SPL #13

6
Comments 1
2 min read
loading...