DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Analyzing Airbnb Listings in Chicago: A Power BI Dashboard Project
raj_tiwari_f987064d2f1827 profile

Analyzing Airbnb Listings in Chicago: A Power BI Dashboard Project

#datascience #dataengineering #data
1
Comments
4 min read
Intro to SQL using Apache Iceberg and Dremio
alexmercedcoder profile

Intro to SQL using Apache Iceberg and Dremio

#database #sql #dataengineering #datascience
4
Comments
22 min read
5 Best ETL Tools: A Comprehensive Comparison Guide
techsourabh profile

5 Best ETL Tools: A Comprehensive Comparison Guide

#etl #datascience #dataengineering #learning
1
Comments
3 min read
Data Engineering with Scala: Mastering Real-Time Data Processing with Apache Flink and Google Pub/Sub
geazi_anc profile

Data Engineering with Scala: Mastering Real-Time Data Processing with Apache Flink and Google Pub/Sub

#dataengineering #scala #datascience #flink
1
Comments
15 min read
SAP S/4HANA Cloud
dentrodailha96 profile
Hiromi

SAP S/4HANA Cloud

#sap #dataengineering #erp #cloud
Comments 1
2 min read
Why Apache Spark RDD is immutable?
luminousmen profile

Why Apache Spark RDD is immutable?

#dataengineering #bigdata #data
Comments
3 min read
Data Modeling - Entities and Events
alexmercedcoder profile

Data Modeling - Entities and Events

#database #dataengineering #datamodeling #dataanalytics
1
Comments
6 min read
Data Engineering in Observability: The Backbone of Modern Monitoring
awa_emma profile

Data Engineering in Observability: The Backbone of Modern Monitoring

#dataengineering #observability #opensource #banking
1
Comments
5 min read
Análise de dados de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka
geazi_anc profile

Análise de dados de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka

#dataengineering #python #braziliandevs #spark
1
Comments
8 min read
Oracle to Snowflake Migration: Steps, Challenges & Best Practices
techsourabh profile

Oracle to Snowflake Migration: Steps, Challenges & Best Practices

#datascience #dataengineering #tutorial #learning
1
Comments
3 min read
Data Engineering in 2024: Innovations and Trends Shaping the Future
missmati profile

Data Engineering in 2024: Innovations and Trends Shaping the Future

#dataengineering #data #analytics
6
Comments 2
13 min read
My journey learning Apache Spark
pauletart profile

My journey learning Apache Spark

#spark #sql #dataengineering
1
Comments
2 min read
AWS DATA ENGINEER - 101
sajjadrahman56 profile

AWS DATA ENGINEER - 101

#aws #dataengineering #awschallenge #awsbigdata
3
Comments
2 min read
The Journey From a CSV File to Apache Hive Table
abdullah_haggag profile

The Journey From a CSV File to Apache Hive Table

#hadoop #hive #bigdata #dataengineering
5
Comments
6 min read
Achieving Clean and Scalable PySpark Code: A Guide to Avoiding Redundancy
gugomes profile

Achieving Clean and Scalable PySpark Code: A Guide to Avoiding Redundancy

#pyspark #dataengineering #cleancode #python
Comments
5 min read
Capítulo 2 - Modelos de Datos y Lenguajes de Consulta
pabloar profile

Capítulo 2 - Modelos de Datos y Lenguajes de Consulta

#data #sql #nosql #dataengineering
2
Comments
7 min read
All About Parquet Part 01 - An Introduction
alexmercedcoder profile

All About Parquet Part 01 - An Introduction

#database #dataengineering
1
Comments
4 min read
All About Parquet Part 05 - Compression Techniques in Parquet
alexmercedcoder profile

All About Parquet Part 05 - Compression Techniques in Parquet

#database #datascience #dataengineering
3
Comments
5 min read
All About Parquet Part 10 - Performance Tuning and Best Practices with Parquet
alexmercedcoder profile

All About Parquet Part 10 - Performance Tuning and Best Practices with Parquet

#database #datascience #dataengineering
7
Comments
6 min read
All About Parquet Part 03 - Parquet File Structure | Pages, Row Groups, and Columns
alexmercedcoder profile

All About Parquet Part 03 - Parquet File Structure | Pages, Row Groups, and Columns

#database #datascience #dataengineering
2
Comments
5 min read
All About Parquet Part 08 - Reading and Writing Parquet Files in Python
alexmercedcoder profile

All About Parquet Part 08 - Reading and Writing Parquet Files in Python

#database #datascience #dataengineering #data
9
Comments
5 min read
All About Parquet Part 07 - Metadata in Parquet | Improving Data Efficiency
alexmercedcoder profile

All About Parquet Part 07 - Metadata in Parquet | Improving Data Efficiency

#data #database #dataengineering #datascience
1
Comments
5 min read
All About Parquet Part 04 - Schema Evolution in Parquet
alexmercedcoder profile

All About Parquet Part 04 - Schema Evolution in Parquet

#database #datascience #dataengineering
3
Comments
5 min read
All About Parquet Part 06 - Encoding in Parquet | Optimizing for Storage
alexmercedcoder profile

All About Parquet Part 06 - Encoding in Parquet | Optimizing for Storage

#database #datascience #dataengineering
1
Comments
6 min read
From a Unified Bronze Layer to Multiple Silver Layers: Streamlining Data Transformation in Databricks Unity Catalog
prakhyatkarri profile

From a Unified Bronze Layer to Multiple Silver Layers: Streamlining Data Transformation in Databricks Unity Catalog

#databricks #unitycatalog #medallionarchitecture #dataengineering
2
Comments
5 min read
Clear Link Between DevSecOps and Data Engineering
regnard profile

Clear Link Between DevSecOps and Data Engineering

#dataengineering #devops #devsecops #cloud
Comments
1 min read
Still Using SQL, Python, & Excel for Data Deduplication? Here's Why You Need Better Tools.
farahkim profile

Still Using SQL, Python, & Excel for Data Deduplication? Here's Why You Need Better Tools.

#algorithms #ai #dataengineering
5
Comments
4 min read
Building a Big Data Playground Sandbox for Learning
abdullah_haggag profile

Building a Big Data Playground Sandbox for Learning

#dataengineering #bigdata #opensource
5
Comments
5 min read
Capture Browser XHR/Fetch API Response Automatically into JSON Files
dendihandian profile

Capture Browser XHR/Fetch API Response Automatically into JSON Files

#help #dataengineering #chrome #javascript
Comments
1 min read
The True Cost of Poor Data Quality: Why It Matters and How to Improve It
markyu profile

The True Cost of Poor Data Quality: Why It Matters and How to Improve It

#database #datascience #dataengineering #management
3
Comments
6 min read
Explaining the History of Data Lakehouse
pkutaj profile

Explaining the History of Data Lakehouse

#lakehouse #dataengineering #warehouse
Comments
2 min read
Building a User-Friendly, Budget-Friendly Alternative to dbt Cloud
mporracin profile

Building a User-Friendly, Budget-Friendly Alternative to dbt Cloud

#dbt #dataengineering #opensource #datascience
Comments
1 min read
O que é Engenharia de Dados?
norton794 profile

O que é Engenharia de Dados?

#dataengineering #datascience
3
Comments
1 min read
How SQL Spatial Data Solves Real-World Problems
nuthankishore profile

How SQL Spatial Data Solves Real-World Problems

#firstpost #spatialdata #dataengineering
Comments
6 min read
Explaining CDC (Change Data Capture)
pkutaj profile

Explaining CDC (Change Data Capture)

#databricks #dataengineering
Comments
1 min read
Handling Outliers 101: Why the IQR Method is Your Go-To Tool
allan-pg profile

Handling Outliers 101: Why the IQR Method is Your Go-To Tool

#python #datascience #dataengineering #data
2
Comments
3 min read
Go vs Python for File Processing: A Performance and Architecture Perspective
nicobistolfi profile

Go vs Python for File Processing: A Performance and Architecture Perspective

#python #go #performance #dataengineering
2
Comments 2
5 min read
Exploring Data Operations with PySpark, Pandas, DuckDB, Polars, and DataFusion in a Python Notebook
alexmercedcoder profile

Exploring Data Operations with PySpark, Pandas, DuckDB, Polars, and DataFusion in a Python Notebook

#python #database #datascience #dataengineering
4
Comments
13 min read
Secure Data Stack: Navigating Adoption Challenges of Data Encryption
l1990790120 profile

Secure Data Stack: Navigating Adoption Challenges of Data Encryption

#security #dataengineering #encryption #infosec
1
Comments
5 min read
Python 101: Introduction to Python as a Data Analytics Tool
edvichuki profile

Python 101: Introduction to Python as a Data Analytics Tool

#python #analytics #datascience #dataengineering
Comments
3 min read
Ultimate Directory of Apache Iceberg Resources
alexmercedcoder profile

Ultimate Directory of Apache Iceberg Resources

#database #dataengineering #datascience #elasticsearch
1
Comments
14 min read
Understanding OLTP and Choosing the Right Database
chaets profile

Understanding OLTP and Choosing the Right Database

#dataengineering #mongodb #postgressql #mysql
1
Comments
6 min read
Change Data Capture (CDC) when there is no CDC
alexmercedcoder profile

Change Data Capture (CDC) when there is no CDC

#database #dataengineering #postgres
1
Comments
11 min read
The Ultimate Guide to Data Engineering
milcah profile

The Ultimate Guide to Data Engineering

#dataengineering #data
Comments
2 min read
Evolution of Data Sharding Towards Automation and Flexibility
apachedoris profile

Evolution of Data Sharding Towards Automation and Flexibility

#opensource #dataengineering #database #automation
Comments
15 min read
Data Showdown: OLAP vs. OLTP – The Battle of Real-Time and Analytics Titans
chaets profile

Data Showdown: OLAP vs. OLTP – The Battle of Real-Time and Analytics Titans

#bigdata #dataengineering #understanding #database
Comments
5 min read
Serverless PDF Processing with AWS Lambda and Textract
olasha profile

Serverless PDF Processing with AWS Lambda and Textract

#cloudcomputing #serverless #lambda #dataengineering
10
Comments 1
9 min read
The Simplest Data Architecture
panasenco profile

The Simplest Data Architecture

#data #architecture #dataengineering #analytics
1
Comments
21 min read
🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)
danc profile

🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)

#database #mongodb #data #dataengineering
Comments
2 min read
ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization
arooba_aqeel_b96d53915b06 profile

ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization

#dataengineering #etlproject #aws
Comments
3 min read
End-to-End AWS KMS Encryption and Decryption Tutorial
l1990790120 profile

End-to-End AWS KMS Encryption and Decryption Tutorial

#tutorial #aws #kms #dataengineering
2
Comments
3 min read
Working with Gigantic Google BigQuery Partitioned Tables in DBT
stephen84s profile

Working with Gigantic Google BigQuery Partitioned Tables in DBT

#bigquery #dbt #dataengineering #googlecloud
2
Comments
3 min read
Cogumelos Mágicos: explorando e tratando dados nulos com Mage
deadpunnk profile

Cogumelos Mágicos: explorando e tratando dados nulos com Mage

#python #mage #etl #dataengineering
Comments
6 min read
Apache Airflow
sandeepk27 profile

Apache Airflow

#airflow #development #workflow #dataengineering
2
Comments
4 min read
Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting
anshul_kichara profile

Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting

#dataengineering #devops #technology #software
Comments
2 min read
The Must-Have Features of Modern Data Transformation Tools
paula_david_dataforg profile

The Must-Have Features of Modern Data Transformation Tools

#datatransformation #datatransformationtools #dataengineering #bigdata
Comments
6 min read
An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example
chaets profile

An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example

#dataengineering #datatransformation #dbt #bigdata
4
Comments
4 min read
Data Pipeline Techniques in Action
bugsbunnyshah profile

Data Pipeline Techniques in Action

#opensource #datascience #softwaredevelopment #dataengineering
1
Comments
1 min read
From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics
jinesh_vora_ab4d7886e6a8d profile

From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics

#datascience #database #dataengineering #python
1
Comments
8 min read
Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey
elnino_impact profile

Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey

#twitter #dataengineering #socialmedia #startup
1
Comments 1
3 min read
loading...