DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example
chaets profile

An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example

#dataengineering #datatransformation #dbt #bigdata
Comments
4 min read
Apache Airflow
sandeepk27 profile

Apache Airflow

#airflow #development #workflow #dataengineering
1
Comments
4 min read
Clear Link Between DevSecOps and Data Engineering
regnard profile

Clear Link Between DevSecOps and Data Engineering

#dataengineering #devops #devsecops #cloud
Comments
1 min read
Capture Browser XHR/Fetch API Response Automatically into JSON Files
dendihandian profile

Capture Browser XHR/Fetch API Response Automatically into JSON Files

#help #dataengineering #chrome #javascript
Comments
1 min read
One Minute: DatAasee
gramian profile

One Minute: DatAasee

#oneminute #dataengineering #database #metadatalake
Comments
1 min read
Building a User-Friendly, Budget-Friendly Alternative to dbt Cloud
mporracin profile

Building a User-Friendly, Budget-Friendly Alternative to dbt Cloud

#dbt #dataengineering #opensource #datascience
Comments
1 min read
Ensuring Data Integrity: Comparing Soda and Great Expectations for Quality Assurance
alfianpr profile

Ensuring Data Integrity: Comparing Soda and Great Expectations for Quality Assurance

#data #dataengineering #python
Comments
4 min read
Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey
elnino_impact profile

Building Powerful Social Media APIs for Twitter and Telegram: A Developer's Journey

#twitter #dataengineering #socialmedia #startup
1
Comments
3 min read
How SQL Spatial Data Solves Real-World Problems
nuthankishore profile

How SQL Spatial Data Solves Real-World Problems

#firstpost #spatialdata #dataengineering
Comments
6 min read
Secure Data Stack: Navigating Adoption Challenges of Data Encryption
l1990790120 profile

Secure Data Stack: Navigating Adoption Challenges of Data Encryption

#security #dataengineering #encryption #infosec
1
Comments
5 min read
The Ultimate Guide to Data Engineering
milcah profile

The Ultimate Guide to Data Engineering

#dataengineering #data
Comments
2 min read
Understanding the Apache Iceberg Manifest File
alexmercedcoder profile

Understanding the Apache Iceberg Manifest File

#dataengineering #dataanalytics
Comments
7 min read
Evolution of Data Sharding Towards Automation and Flexibility
apachedoris profile

Evolution of Data Sharding Towards Automation and Flexibility

#opensource #dataengineering #database #automation
Comments
15 min read
The Power of Data Analytics – Transforming Businesses with Insights
ismailco96 profile

The Power of Data Analytics – Transforming Businesses with Insights

#dataengineering #datascience #startup
Comments
5 min read
Top 5 Things You Should Know About Spark
sudo_pradip profile

Top 5 Things You Should Know About Spark

#spark #dataengineering #development #coding
1
Comments
3 min read
Understanding Apache Iceberg's metadata.json file
alexmercedcoder profile

Understanding Apache Iceberg's metadata.json file

#data #dataengineering #database
1
Comments
7 min read
Strategy Recall Platform
lips7 profile

Strategy Recall Platform

#python #dataengineering #performance #fullstack
Comments
13 min read
🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)
danc profile

🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)

#database #mongodb #data #dataengineering
Comments
2 min read
ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization
arooba_aqeel_b96d53915b06 profile

ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization

#dataengineering #etlproject #aws
Comments
3 min read
End-to-End AWS KMS Encryption and Decryption Tutorial
l1990790120 profile

End-to-End AWS KMS Encryption and Decryption Tutorial

#tutorial #aws #kms #dataengineering
2
Comments
3 min read
Cogumelos Mágicos: explorando e tratando dados nulos com Mage
deadpunnk profile

Cogumelos Mágicos: explorando e tratando dados nulos com Mage

#python #mage #etl #dataengineering
Comments
6 min read
Understanding Apache Iceberg Delete Files
alexmercedcoder profile

Understanding Apache Iceberg Delete Files

#apacheiceberg #dataengineering #database #datascience
1
Comments
4 min read
Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting
anshul_kichara profile

Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting

#dataengineering #devops #technology #software
Comments
2 min read
The Must-Have Features of Modern Data Transformation Tools
paula_david_dataforg profile

The Must-Have Features of Modern Data Transformation Tools

#datatransformation #datatransformationtools #dataengineering #bigdata
Comments
6 min read
Data Pipeline Techniques in Action
bugsbunnyshah profile

Data Pipeline Techniques in Action

#opensource #datascience #softwaredevelopment #dataengineering
1
Comments
1 min read
From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics
jinesh_vora_ab4d7886e6a8d profile

From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics

#datascience #database #dataengineering #python
1
Comments
8 min read
Useful Python Libraries for AI/ML
johnscode profile

Useful Python Libraries for AI/ML

#machinelearning #ai #datascience #dataengineering
Comments
1 min read
🦆 💏 🐘 Let PostgreSQL & duckdb "sql" together
adriens profile

🦆 💏 🐘 Let PostgreSQL & duckdb "sql" together

#database #postgres #dataengineering #tutorial
Comments 2
3 min read
Data Security Strategy Beyond Access Control: Data Encryption
l1990790120 profile

Data Security Strategy Beyond Access Control: Data Encryption

#security #dataengineering #encryption #infosec
2
Comments
5 min read
A beginner's guide to data engineering concepts, tools, and responsibilities.
john_otienoh profile

A beginner's guide to data engineering concepts, tools, and responsibilities.

#dataengineering #data #datascience
Comments
1 min read
Avoid These Top 10 Mistakes When Using Apache Spark
sudo_pradip profile

Avoid These Top 10 Mistakes When Using Apache Spark

#spark #dataengineering #development #coding
4
Comments
8 min read
A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.
rebeccacheptoek profile

A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.

#bigdata #dataengineering
Comments
1 min read
Snowflake vs. BigQuery: Choosing the Right Cloud Platform for Your Data
onixcloud profile

Snowflake vs. BigQuery: Choosing the Right Cloud Platform for Your Data

#cloud #googlecloud #bigquery #dataengineering
Comments
2 min read
Building a data science career as a beginner. How can you do it?
mugultum profile

Building a data science career as a beginner. How can you do it?

#datascience #beginners #newbie #dataengineering
Comments
4 min read
Hiring Alert!
dorothy_seal_632ae57b1802 profile

Hiring Alert!

#dataengineering #pyspark #python #aws
Comments
1 min read
Uploading Files Using Pre-Signed URLs to a Specific Storage Class
anshul_kichara profile

Uploading Files Using Pre-Signed URLs to a Specific Storage Class

#devops #dataengineering #software #technology
Comments
2 min read
PySpark optimization techniques
rado_mayank profile

PySpark optimization techniques

#pyspark #dataengineering #spark #optimization
1
Comments
4 min read
Data Engineer and Databricks
christiano profile

Data Engineer and Databricks

#dataengineering #data
Comments
3 min read
Getting Started with Apache Kafka: A Beginner's Guide to Distributed Event Streaming
amitchandra profile

Getting Started with Apache Kafka: A Beginner's Guide to Distributed Event Streaming

#apachekafka #distributedsystems #eventstreaming #dataengineering
1
Comments
5 min read
Unlocking the Potential of Data with Azure Data Engineers
pangaea_x profile

Unlocking the Potential of Data with Azure Data Engineers

#azuredataengineers #dataengineering #beginners #learning
1
Comments
3 min read
RoadMap to Data-Analytics 2024!
parthmagicss profile

RoadMap to Data-Analytics 2024!

#datascience #dataengineering #sql #database
3
Comments
2 min read
DBT and Software Engineering
sudo_pradip profile

DBT and Software Engineering

#dbt #softwareengineering #dataengineering #learning
3
Comments
7 min read
Effective Techniques for Handling Imbalanced Datasets: My Proven Approach
dev_dive profile

Effective Techniques for Handling Imbalanced Datasets: My Proven Approach

#ai #datascience #dataengineering
Comments
3 min read
The Developer’s Guide to Real-Time Data Platforms!
pavanbelagatti profile

The Developer’s Guide to Real-Time Data Platforms!

#datascience #dataengineering #database #developers
7
Comments
6 min read
🌐 Get started: What is MongoDB operational data layer? (Part 2) 🌐
danc profile

🌐 Get started: What is MongoDB operational data layer? (Part 2) 🌐

#database #data #dataengineering #mongodb
5
Comments
2 min read
🌐 开始使用: MongoDB Operational Data Layer 是什么? (第1部分)
danc profile

🌐 开始使用: MongoDB Operational Data Layer 是什么? (第1部分)

#data #database #mongodb #dataengineering
5
Comments
1 min read
Mastering SQL Joins and Unions: Integrate Data for Incredible Insights
jinesh_vora_ab4d7886e6a8d profile

Mastering SQL Joins and Unions: Integrate Data for Incredible Insights

#datascience #dataengineering #machinelearning #mysql
Comments
6 min read
Feature Engineering: The Ultimate Guide
anna_m profile

Feature Engineering: The Ultimate Guide

#dataengineering #machinelearning #featureengineering
1
Comments
2 min read
What Apache Iceberg REST Catalog is and isn't
alexmercedcoder profile

What Apache Iceberg REST Catalog is and isn't

#dataengineering #datascience
9
Comments
3 min read
Transforming Data Engineering: A Business Domain Approach with Data Mesh
alfianpr profile

Transforming Data Engineering: A Business Domain Approach with Data Mesh

#data #business #database #dataengineering
Comments
5 min read
Speeding Up Data on AWS: From Ingestion to Insights
pmalmirae profile

Speeding Up Data on AWS: From Ingestion to Insights

#dataengineering #aws #awsdatalake #etl
4
Comments
11 min read
การนำเข้าข้อมูลจากไฟล์ CSV เข้ามาใน Posstgres : ทักษะเบื้องต้นของ Data Engineer
iconnext profile

การนำเข้าข้อมูลจากไฟล์ CSV เข้ามาใน Posstgres : ทักษะเบื้องต้นของ Data Engineer

#postgres #csv #dataengineering
Comments
1 min read
The Ultimate Guide to Data Analytics: Techniques and Tools.
s_ndichu profile

The Ultimate Guide to Data Analytics: Techniques and Tools.

#datascience #analytics #data #dataengineering
Comments
3 min read
Building an Agnostic Data Pipeline: Pros and Cons
alfianpr profile

Building an Agnostic Data Pipeline: Pros and Cons

#data #database #dataengineering
Comments
4 min read
🐚 My Pacific Dataviz Challenge 2024 submission : violence & graphdatascience
adriens profile

🐚 My Pacific Dataviz Challenge 2024 submission : violence & graphdatascience

#opendata #datascience #dataengineering #neo4j
3
Comments 9
2 min read
Apache Doris for log and time series data analysis in NetEase, why not Elasticsearch and InfluxDB?
apachedoris profile

Apache Doris for log and time series data analysis in NetEase, why not Elasticsearch and InfluxDB?

#datascience #database #dataengineering #opensource
Comments
9 min read
Understanding RAID Levels: A Comprehensive Guide to RAID 0, 1, 5, 6, 10, and Beyond
pltnvs profile

Understanding RAID Levels: A Comprehensive Guide to RAID 0, 1, 5, 6, 10, and Beyond

#dataengineering #softwareraid #dataredundancy #datastorage
6
Comments
9 min read
Understanding Your Data: The Essentials of Exploratory Data Analysis (EDA)
john_analytics profile

Understanding Your Data: The Essentials of Exploratory Data Analysis (EDA)

#datascience #python #dataengineering #sql
1
Comments
3 min read
Engenharia de Dados com Scala: masterizando o processamento de dados em tempo real com Apache Flink e Google Pub/Sub
geazi_anc profile

Engenharia de Dados com Scala: masterizando o processamento de dados em tempo real com Apache Flink e Google Pub/Sub

#dataengineering #scala #braziliandevs #gcp
5
Comments
16 min read
Data Lakehouse 101: The Who, What and Why of Data Lakehouses
alexmercedcoder profile

Data Lakehouse 101: The Who, What and Why of Data Lakehouses

#data #database #dataengineering #apacheiceberg
Comments
7 min read
loading...