DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
The Simplest Data Architecture
panasenco profile

The Simplest Data Architecture

#data #architecture #dataengineering #analytics
1
Comments
21 min read
🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)
danc profile

🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)

#database #mongodb #data #dataengineering
Comments
2 min read
ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization
arooba_aqeel_b96d53915b06 profile

ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization

#dataengineering #etlproject #aws
Comments
3 min read
End-to-End AWS KMS Encryption and Decryption Tutorial
l1990790120 profile

End-to-End AWS KMS Encryption and Decryption Tutorial

#tutorial #aws #kms #dataengineering
2
Comments
3 min read
Working with Gigantic Google BigQuery Partitioned Tables in DBT
stephen84s profile

Working with Gigantic Google BigQuery Partitioned Tables in DBT

#bigquery #dbt #dataengineering #googlecloud
2
Comments
3 min read
Cogumelos MĂĄgicos: explorando e tratando dados nulos com Mage
deadpunnk profile

Cogumelos MĂĄgicos: explorando e tratando dados nulos com Mage

#python #mage #etl #dataengineering
Comments
6 min read
Apache Airflow
sandeepk27 profile

Apache Airflow

#airflow #development #workflow #dataengineering
2
Comments
4 min read
Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting
anshul_kichara profile

Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting

#dataengineering #devops #technology #software
Comments
2 min read
The Must-Have Features of Modern Data Transformation Tools
paula_david_dataforg profile

The Must-Have Features of Modern Data Transformation Tools

#datatransformation #datatransformationtools #dataengineering #bigdata
Comments
6 min read
An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example
chaets profile

An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example

#dataengineering #datatransformation #dbt #bigdata
4
Comments
4 min read
Data Pipeline Techniques in Action
bugsbunnyshah profile

Data Pipeline Techniques in Action

#opensource #datascience #softwaredevelopment #dataengineering
1
Comments
1 min read
From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics
jinesh_vora_ab4d7886e6a8d profile

From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics

#datascience #database #dataengineering #python
1
Comments
8 min read
Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey
elnino_impact profile

Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey

#twitter #dataengineering #socialmedia #startup
1
Comments 1
3 min read
One Minute: DatAasee
gramian profile

One Minute: DatAasee

#oneminute #dataengineering #database #metadatalake
1
Comments
1 min read
Data Security Strategy Beyond Access Control: Data Encryption
l1990790120 profile

Data Security Strategy Beyond Access Control: Data Encryption

#security #dataengineering #encryption #infosec
2
Comments
5 min read
The Power of Data Analytics – Transforming Businesses with Insights
ismailco96 profile

The Power of Data Analytics – Transforming Businesses with Insights

#dataengineering #datascience #startup
Comments
5 min read
A beginner's guide to data engineering concepts, tools, and responsibilities.
john_otienoh profile

A beginner's guide to data engineering concepts, tools, and responsibilities.

#dataengineering #data #datascience
Comments
1 min read
Ensuring Data Integrity: Comparing Soda and Great Expectations for Quality Assurance
alfianpr profile

Ensuring Data Integrity: Comparing Soda and Great Expectations for Quality Assurance

#data #dataengineering #python
4
Comments
4 min read
A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.
rebeccacheptoek profile

A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.

#bigdata #dataengineering
Comments
1 min read
Snowflake vs. BigQuery: Choosing the Right Cloud Platform for Your Data
onixcloud profile

Snowflake vs. BigQuery: Choosing the Right Cloud Platform for Your Data

#cloud #googlecloud #bigquery #dataengineering
Comments
2 min read
Building a data science career as a beginner. How can you do it?
mugultum profile

Building a data science career as a beginner. How can you do it?

#datascience #beginners #newbie #dataengineering
Comments
4 min read
Hiring Alert!
dorothy_seal_632ae57b1802 profile

Hiring Alert!

#dataengineering #pyspark #python #aws
Comments
1 min read
Understanding Apache Iceberg Delete Files
alexmercedcoder profile

Understanding Apache Iceberg Delete Files

#apacheiceberg #dataengineering #database #datascience
4
Comments
4 min read
Top 5 Things You Should Know About Spark
sudo_pradip profile

Top 5 Things You Should Know About Spark

#spark #dataengineering #development #coding
1
Comments
3 min read
Uploading Files Using Pre-Signed URLs to a Specific Storage Class
anshul_kichara profile

Uploading Files Using Pre-Signed URLs to a Specific Storage Class

#devops #dataengineering #software #technology
Comments
2 min read
PySpark optimization techniques
rado_mayank profile

PySpark optimization techniques

#pyspark #dataengineering #spark #optimization
1
Comments
4 min read
Avoid These Top 10 Mistakes When Using Apache Spark
sudo_pradip profile

Avoid These Top 10 Mistakes When Using Apache Spark

#dataengineering #development #coding #dbt
3
Comments
8 min read
Getting Started with Apache Kafka: A Beginner's Guide to Distributed Event Streaming
amitchandra profile

Getting Started with Apache Kafka: A Beginner's Guide to Distributed Event Streaming

#apachekafka #distributedsystems #eventstreaming #dataengineering
1
Comments
5 min read
Understanding the Apache Iceberg Manifest File
alexmercedcoder profile

Understanding the Apache Iceberg Manifest File

#dataengineering #dataanalytics
3
Comments
7 min read
RoadMap to Data-Analytics 2024!
parthmagicss profile

RoadMap to Data-Analytics 2024!

#datascience #dataengineering #sql #database
3
Comments
2 min read
DBT and Software Engineering
sudo_pradip profile

DBT and Software Engineering

#dbt #softwareengineering #dataengineering #learning
4
Comments
7 min read
Effective Techniques for Handling Imbalanced Datasets: My Proven Approach
dev_dive profile

Effective Techniques for Handling Imbalanced Datasets: My Proven Approach

#ai #datascience #dataengineering
Comments
3 min read
The Developer’s Guide to Real-Time Data Platforms!
pavanbelagatti profile

The Developer’s Guide to Real-Time Data Platforms!

#datascience #dataengineering #database #developers
9
Comments
6 min read
Understanding Apache Iceberg's metadata.json file
alexmercedcoder profile

Understanding Apache Iceberg's metadata.json file

#data #dataengineering #database
4
Comments
7 min read
🌐 开始使用: MongoDB Operational Data Layer 是什么? (第1部分)
danc profile

🌐 开始使用: MongoDB Operational Data Layer 是什么? (第1部分)

#data #database #mongodb #dataengineering
5
Comments
1 min read
🌐 Get started: What is MongoDB operational data layer? (Part 2) 🌐
danc profile

🌐 Get started: What is MongoDB operational data layer? (Part 2) 🌐

#database #data #dataengineering #mongodb
5
Comments
2 min read
Mastering SQL Joins and Unions: Integrate Data for Incredible Insights
jinesh_vora_ab4d7886e6a8d profile

Mastering SQL Joins and Unions: Integrate Data for Incredible Insights

#datascience #dataengineering #machinelearning #mysql
Comments
6 min read
Feature Engineering: The Ultimate Guide
anna_m profile

Feature Engineering: The Ultimate Guide

#dataengineering #machinelearning #featureengineering
1
Comments
2 min read
🦆 💏 🐘 Let PostgreSQL & duckdb "sql" together
adriens profile

🦆 💏 🐘 Let PostgreSQL & duckdb "sql" together

#database #postgres #dataengineering #tutorial
2
Comments 2
3 min read
What Apache Iceberg REST Catalog is and isn't
alexmercedcoder profile

What Apache Iceberg REST Catalog is and isn't

#dataengineering #datascience
10
Comments
3 min read
Transforming Data Engineering: A Business Domain Approach with Data Mesh
alfianpr profile

Transforming Data Engineering: A Business Domain Approach with Data Mesh

#data #business #database #dataengineering
Comments
5 min read
Speeding Up Data on AWS: From Ingestion to Insights
pmalmirae profile

Speeding Up Data on AWS: From Ingestion to Insights

#dataengineering #aws #awsdatalake #etl
4
Comments
11 min read
การนำเข้าข้อมูลจากไฟล์ CSV เข้ามาใน Posstgres : ทักษะเบื้องต้นของ Data Engineer
iconnext profile

การนำเข้าข้อมูลจากไฟล์ CSV เข้ามาใน Posstgres : ทักษะเบื้องต้นของ Data Engineer

#postgres #csv #dataengineering
Comments
1 min read
The Ultimate Guide to Data Analytics: Techniques and Tools.
s_ndichu profile

The Ultimate Guide to Data Analytics: Techniques and Tools.

#datascience #analytics #data #dataengineering
Comments
3 min read
Building an Agnostic Data Pipeline: Pros and Cons
alfianpr profile

Building an Agnostic Data Pipeline: Pros and Cons

#data #database #dataengineering
1
Comments
4 min read
🐚 My Pacific Dataviz Challenge 2024 submission : violence & graphdatascience
adriens profile

🐚 My Pacific Dataviz Challenge 2024 submission : violence & graphdatascience

#opendata #datascience #dataengineering #neo4j
3
Comments 10
2 min read
Useful Python Libraries for AI/ML
johnscode profile

Useful Python Libraries for AI/ML

#machinelearning #ai #datascience #dataengineering
2
Comments
1 min read
Understanding RAID Levels: A Comprehensive Guide to RAID 0, 1, 5, 6, 10, and Beyond
pltnvs profile

Understanding RAID Levels: A Comprehensive Guide to RAID 0, 1, 5, 6, 10, and Beyond

#dataengineering #softwareraid #dataredundancy #datastorage
6
Comments
9 min read
Engenharia de Dados com Scala: masterizando o processamento de dados em tempo real com Apache Flink e Google Pub/Sub
geazi_anc profile

Engenharia de Dados com Scala: masterizando o processamento de dados em tempo real com Apache Flink e Google Pub/Sub

#dataengineering #scala #braziliandevs #flink
11
Comments
16 min read
Understanding Your Data: The Essentials of Exploratory Data Analysis (EDA)
john_analytics profile

Understanding Your Data: The Essentials of Exploratory Data Analysis (EDA)

#datascience #python #dataengineering #sql
1
Comments
3 min read
Data Lakehouse 101: The Who, What and Why of Data Lakehouses
alexmercedcoder profile

Data Lakehouse 101: The Who, What and Why of Data Lakehouses

#data #database #dataengineering #apacheiceberg
Comments
7 min read
Elasticsearch: Finding Missing Documents between 2 indices
aditya_singh_78be52d82a3a profile

Elasticsearch: Finding Missing Documents between 2 indices

#elasticsearch #dataengineering
3
Comments
3 min read
Breaking Into Data Science: A Comprehensive Guide for Aspiring Data Scientists
nderitugichuki profile

Breaking Into Data Science: A Comprehensive Guide for Aspiring Data Scientists

#datascience #dataengineering #devops #machinelearning
Comments
5 min read
"Data Engineering 101: A Beginner's Guide"
samtheanalyst profile

"Data Engineering 101: A Beginner's Guide"

#datascience #dataengineering #codenewbie
3
Comments
3 min read
Understanding the Polaris Iceberg Catalog and Its Architecture
alexmercedcoder profile

Understanding the Polaris Iceberg Catalog and Its Architecture

#database #dataengineering #apacheiceberg #polaris
2
Comments
8 min read
Automatically Update BigQuery View Schema Changes
matthieucham profile

Automatically Update BigQuery View Schema Changes

#bigquery #eventdriven #googlecloud #dataengineering
3
Comments
5 min read
How I contributed my first data pipeline to the open source.
aman_gupta_7c59c96e9e167a profile

How I contributed my first data pipeline to the open source.

#dataengineering #etl #data #pipeline
1
Comments
3 min read
On Orchestrators: You Are All Right, But You Are All Wrong Too
aman_gupta_7c59c96e9e167a profile

On Orchestrators: You Are All Right, But You Are All Wrong Too

#dataengineering #etl #pipeline
1
Comments
10 min read
From Messy Data to Super Mario Pipeline: My First Adventure in Data Engineering
jampamatos profile

From Messy Data to Super Mario Pipeline: My First Adventure in Data Engineering

#dataengineering #python #automation #sql
Comments
12 min read
Data Engineer and Databricks
christiano profile

Data Engineer and Databricks

#dataengineering #data
1
Comments
3 min read
loading...