Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Search
Log in
Create account
DEV Community
Close
#
bigdata
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
🚀 Unlock the Power of ORC File Format 📊
Pratik Barjatiya
Pratik Barjatiya
Pratik Barjatiya
Follow
Nov 22
🚀 Unlock the Power of ORC File Format 📊
#
dataengineering
#
bigdata
#
datascience
#
data
Comments
Add Comment
1 min read
The Heart of DolphinScheduler: In-Depth Analysis of the Quartz Scheduling Framework
Chen Debra
Chen Debra
Chen Debra
Follow
Nov 20
The Heart of DolphinScheduler: In-Depth Analysis of the Quartz Scheduling Framework
#
apachedolphinscheduler
#
quartz
#
opensource
#
bigdata
8
reactions
Comments
Add Comment
3 min read
Big Data
williamxlr
williamxlr
williamxlr
Follow
Nov 13
Big Data
#
bigdata
#
hadoop
#
spark
Comments
Add Comment
1 min read
System Design 09 - Data Partitioning: Dividing to Conquer Big Data
Sarva Bharan
Sarva Bharan
Sarva Bharan
Follow
Nov 12
System Design 09 - Data Partitioning: Dividing to Conquer Big Data
#
systemdesign
#
bigdata
#
datapartition
Comments
Add Comment
2 min read
Simplifying Real-Time Data Ingestion with Apache NiFi
Noel Campbell
Noel Campbell
Noel Campbell
Follow
Nov 6
Simplifying Real-Time Data Ingestion with Apache NiFi
#
nifi
#
bigdata
#
webdev
Comments
Add Comment
3 min read
Understanding Star Schema vs. Snowflake Schema
Puneet Verma
Puneet Verma
Puneet Verma
Follow
Nov 16
Understanding Star Schema vs. Snowflake Schema
#
dataengineering
#
datascience
#
datamodeling
#
bigdata
Comments
Add Comment
1 min read
SeaTunnel-Powered Data Integration: How 58 Group Handles Over 500 Billion+ Data Points Daily
Apache SeaTunnel
Apache SeaTunnel
Apache SeaTunnel
Follow
Nov 20
SeaTunnel-Powered Data Integration: How 58 Group Handles Over 500 Billion+ Data Points Daily
#
datascience
#
apacheseatunnel
#
opensource
#
bigdata
5
reactions
Comments
1
comment
5 min read
Best Practices for Data Security in Big Data Projects
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Follow
Oct 24
Best Practices for Data Security in Big Data Projects
#
bestpractices
#
bigdata
#
datasecurity
Comments
Add Comment
6 min read
5 Big Data Use Cases that Retailers Fail to Use for Actionable Insights
Dmytro Spilka
Dmytro Spilka
Dmytro Spilka
Follow
Oct 16
5 Big Data Use Cases that Retailers Fail to Use for Actionable Insights
#
bigdata
Comments
Add Comment
3 min read
From ETL and ELT to Reverse ETL
luminousmen
luminousmen
luminousmen
Follow
Oct 15
From ETL and ELT to Reverse ETL
#
dataengineering
#
bigdata
#
data
Comments
Add Comment
4 min read
Introduction to Big Data Analysis
Madhav Ganesan
Madhav Ganesan
Madhav Ganesan
Follow
Nov 17
Introduction to Big Data Analysis
#
bigdata
#
aws
#
hadoop
#
coding
8
reactions
Comments
Add Comment
13 min read
How Big Data is Powering the Internet of Things (IoT) Revolution - MasTech InfoTrellis
Hana Sato
Hana Sato
Hana Sato
Follow
Oct 14
How Big Data is Powering the Internet of Things (IoT) Revolution - MasTech InfoTrellis
#
techtalks
#
bigdata
#
iot
#
cloudcomputing
Comments
Add Comment
4 min read
Why Scala is the Best Choice for Big Data Applications: Advantages Over Java and Python
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Follow
Oct 11
Why Scala is the Best Choice for Big Data Applications: Advantages Over Java and Python
#
scala
#
java
#
python
#
bigdata
Comments
Add Comment
6 min read
Processando 20 milhões de registros em menos de 5 segundos com Apache Hive.
Airton Lira junior
Airton Lira junior
Airton Lira junior
Follow
Nov 2
Processando 20 milhões de registros em menos de 5 segundos com Apache Hive.
#
apachehive
#
hive
#
bigdata
#
hadoop
10
reactions
Comments
Add Comment
8 min read
SeaTunnel Community Monthly Report For September
Apache SeaTunnel
Apache SeaTunnel
Apache SeaTunnel
Follow
Oct 9
SeaTunnel Community Monthly Report For September
#
developer
#
apacheseatunnel
#
opensource
#
bigdata
Comments
Add Comment
14 min read
Effizientes Scrapen von JavaScript-Webseiten
hanna Fischer
hanna Fischer
hanna Fischer
Follow
Nov 11
Effizientes Scrapen von JavaScript-Webseiten
#
java
#
python
#
javascript
#
bigdata
Comments
Add Comment
3 min read
Tracking Data Over Time: Slowly Changing Dimensions (SCD)
Chetan Gupta
Chetan Gupta
Chetan Gupta
Follow
Oct 7
Tracking Data Over Time: Slowly Changing Dimensions (SCD)
#
bigdata
#
datatracking
#
scd
#
slowlychangingdimensions
Comments
Add Comment
6 min read
Big Data Challenges and Solutions: Navigating the Complex Landscape
Hana Sato
Hana Sato
Hana Sato
Follow
Oct 4
Big Data Challenges and Solutions: Navigating the Complex Landscape
#
bigdata
#
ai
#
database
#
tutorial
Comments
Add Comment
7 min read
Fünf Schritte zum Scraping mehrerer Bilder mit Python
hanna Fischer
hanna Fischer
hanna Fischer
Follow
Nov 7
Fünf Schritte zum Scraping mehrerer Bilder mit Python
#
python
#
bilder
#
bigdata
#
dataanalyse
Comments
Add Comment
2 min read
Introduction to Big Data
Sourish Srivastava
Sourish Srivastava
Sourish Srivastava
Follow
Nov 2
Introduction to Big Data
#
bigdata
#
ai
#
basic
#
programming
5
reactions
Comments
2
comments
2 min read
Why Apache Spark RDD is immutable?
luminousmen
luminousmen
luminousmen
Follow
Sep 29
Why Apache Spark RDD is immutable?
#
dataengineering
#
bigdata
#
data
Comments
Add Comment
3 min read
Reducing Delivery Times and Costs: How Machine Learning Optimizes Delivery Routes Efficiently
Daryna Mihdal
Daryna Mihdal
Daryna Mihdal
Follow
Oct 30
Reducing Delivery Times and Costs: How Machine Learning Optimizes Delivery Routes Efficiently
#
machinelearning
#
commerce
#
bigdata
1
reaction
Comments
1
comment
3 min read
Hands-on introduction to Apache Iceberg
Claudio Taverna
Claudio Taverna
Claudio Taverna
Follow
for
AWS Community Builders
Oct 28
Hands-on introduction to Apache Iceberg
#
awsdatalake
#
iceberg
#
aws
#
bigdata
7
reactions
Comments
2
comments
8 min read
Embarking on the Big Query Quest: Exploring the Depths of its Inner Workings
Matheus Tramontini
Matheus Tramontini
Matheus Tramontini
Follow
Sep 24
Embarking on the Big Query Quest: Exploring the Depths of its Inner Workings
#
bigquery
#
googlecloud
#
bigdata
#
learning
Comments
Add Comment
5 min read
The Journey From a CSV File to Apache Hive Table
Abdullah Haggag
Abdullah Haggag
Abdullah Haggag
Follow
Oct 24
The Journey From a CSV File to Apache Hive Table
#
hadoop
#
hive
#
bigdata
#
dataengineering
6
reactions
Comments
Add Comment
6 min read
How to Become an Apache SeaTunnel Committer?
Apache SeaTunnel
Apache SeaTunnel
Apache SeaTunnel
Follow
Sep 13
How to Become an Apache SeaTunnel Committer?
#
opensource
#
bigdata
#
datascience
#
apacheseatunnel
1
reaction
Comments
Add Comment
4 min read
Building a Big Data Playground Sandbox for Learning
Abdullah Haggag
Abdullah Haggag
Abdullah Haggag
Follow
Oct 17
Building a Big Data Playground Sandbox for Learning
#
dataengineering
#
bigdata
#
opensource
5
reactions
Comments
Add Comment
5 min read
Big Data Storage Trends and Insights
Mainul Hasan
Mainul Hasan
Mainul Hasan
Follow
Oct 16
Big Data Storage Trends and Insights
#
cloudcomputing
#
cloudstorage
#
bigdata
#
googlecloud
Comments
Add Comment
7 min read
Data Analysis: The Power of Big Data and Analytics in Decision Making 📊
Izabella Albuquerque
Izabella Albuquerque
Izabella Albuquerque
Follow
Oct 16
Data Analysis: The Power of Big Data and Analytics in Decision Making 📊
#
bigdata
#
analytics
#
webdev
#
datascience
Comments
Add Comment
3 min read
Cassandra vs. MongoDB: Choosing the Right NoSQL Database
Noel Campbell
Noel Campbell
Noel Campbell
Follow
Oct 15
Cassandra vs. MongoDB: Choosing the Right NoSQL Database
#
cassandra
#
mongodb
#
nosql
#
bigdata
Comments
Add Comment
3 min read
Which Data Synchronization Method is More Senior?
Apache SeaTunnel
Apache SeaTunnel
Apache SeaTunnel
Follow
Sep 11
Which Data Synchronization Method is More Senior?
#
datascience
#
bigdata
#
seatunnel
#
opensource
1
reaction
Comments
Add Comment
8 min read
Journey Through Spark SQL
Chetan Gupta
Chetan Gupta
Chetan Gupta
Follow
Oct 12
Journey Through Spark SQL
#
sparksql
#
spark
#
bigdata
#
datajourney
Comments
Add Comment
11 min read
Connecting AI with Excel - Talk to Your Spreadsheets
Flowtrail Admin
Flowtrail Admin
Flowtrail Admin
Follow
for
Flowtrail AI
Sep 5
Connecting AI with Excel - Talk to Your Spreadsheets
#
challenge
#
analytics
#
ai
#
bigdata
1
reaction
Comments
Add Comment
6 min read
Scala vs. Java: The Superior Choice for Big Data and Machine Learning
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Aditya Pratap Bhuyan
Follow
Oct 1
Scala vs. Java: The Superior Choice for Big Data and Machine Learning
#
java
#
scala
#
machinelearning
#
bigdata
1
reaction
Comments
1
comment
11 min read
Understanding Data Schemas
Chetan Gupta
Chetan Gupta
Chetan Gupta
Follow
Sep 30
Understanding Data Schemas
#
datawarehouse
#
deltalake
#
bigdata
#
data
Comments
Add Comment
5 min read
The Ultimate Guide to Data Analytics: Unlocking the Power of Data
Samwel Mwangi
Samwel Mwangi
Samwel Mwangi
Follow
Aug 26
The Ultimate Guide to Data Analytics: Unlocking the Power of Data
#
dataanalytics
#
data
#
codenewbie
#
bigdata
Comments
Add Comment
3 min read
Data Showdown: OLAP vs. OLTP – The Battle of Real-Time and Analytics Titans
Chetan Gupta
Chetan Gupta
Chetan Gupta
Follow
Sep 29
Data Showdown: OLAP vs. OLTP – The Battle of Real-Time and Analytics Titans
#
bigdata
#
dataengineering
#
understanding
#
database
Comments
Add Comment
5 min read
Optimize ETL Processes with Apache Iceberg: A Game Changer
Varun Bainsla
Varun Bainsla
Varun Bainsla
Follow
Aug 14
Optimize ETL Processes with Apache Iceberg: A Game Changer
#
iceberg
#
etl
#
awsdatalake
#
bigdata
Comments
Add Comment
4 min read
The Must-Have Features of Modern Data Transformation Tools
Paula David
Paula David
Paula David
Follow
Aug 27
The Must-Have Features of Modern Data Transformation Tools
#
datatransformation
#
datatransformationtools
#
dataengineering
#
bigdata
Comments
Add Comment
6 min read
An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example
Chetan Gupta
Chetan Gupta
Chetan Gupta
Follow
Sep 16
An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example
#
dataengineering
#
datatransformation
#
dbt
#
bigdata
2
reactions
Comments
Add Comment
4 min read
To Index Data is To Sort Data
Judy
Judy
Judy
Follow
Aug 26
To Index Data is To Sort Data
#
sql
#
database
#
bigdata
#
esproc
8
reactions
Comments
Add Comment
5 min read
How to install Apache Kafka on Ubuntu with KRaft Mode (without Zookeeper): A Step-by-Step Guide
Hải Phạm Ngọc
Hải Phạm Ngọc
Hải Phạm Ngọc
Follow
Sep 2
How to install Apache Kafka on Ubuntu with KRaft Mode (without Zookeeper): A Step-by-Step Guide
#
kafka
#
ubuntu
#
bigdata
#
pubsub
Comments
Add Comment
10 min read
Using ReAct Agents LLMs to Draw Insights from Tabular Data
Intelliarts
Intelliarts
Intelliarts
Follow
Sep 11
Using ReAct Agents LLMs to Draw Insights from Tabular Data
#
llm
#
reactagent
#
ai
#
bigdata
6
reactions
Comments
Add Comment
7 min read
Data Driven Dreams: Building My Data Science Career
Adolph Odhiambo
Adolph Odhiambo
Adolph Odhiambo
Follow
Aug 5
Data Driven Dreams: Building My Data Science Career
#
datascience
#
data
#
career
#
bigdata
Comments
Add Comment
4 min read
A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.
Rebeccacheptoek
Rebeccacheptoek
Rebeccacheptoek
Follow
Aug 4
A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.
#
bigdata
#
dataengineering
Comments
Add Comment
1 min read
Optimizing Transformations in Pentaho: Case Study
Phillip L. Cabrera M.
Phillip L. Cabrera M.
Phillip L. Cabrera M.
Follow
Aug 4
Optimizing Transformations in Pentaho: Case Study
#
automation
#
bigdata
#
database
#
career
Comments
Add Comment
3 min read
Loading data to Google Big Query using Dataproc workflow templates and cloud Schedule
Jader Lima
Jader Lima
Jader Lima
Follow
Sep 6
Loading data to Google Big Query using Dataproc workflow templates and cloud Schedule
#
gcp
#
dataproc
#
bigquery
#
bigdata
2
reactions
Comments
Add Comment
12 min read
Data Visualisation Basics
Barbara
Barbara
Barbara
Follow
Sep 6
Data Visualisation Basics
#
datavis
#
python
#
scrollwithme
#
bigdata
9
reactions
Comments
Add Comment
7 min read
Demystifying Data Science: A Beginner’s Guide!
Michelle Njuguna
Michelle Njuguna
Michelle Njuguna
Follow
Aug 4
Demystifying Data Science: A Beginner’s Guide!
#
beginners
#
bigdata
#
ai
#
deeplearning
Comments
Add Comment
3 min read
Data Lakes vs. Data Warehouses: Choosing the Right Big Data Architecture
Haroon Mumtaz
Haroon Mumtaz
Haroon Mumtaz
Follow
Jul 29
Data Lakes vs. Data Warehouses: Choosing the Right Big Data Architecture
#
bigdata
#
architecture
#
businessintelligence
#
softwaredevelopment
1
reaction
Comments
Add Comment
4 min read
How to Install Hadoop on Ubuntu: A Step-by-Step Guide
Hải Phạm Ngọc
Hải Phạm Ngọc
Hải Phạm Ngọc
Follow
Sep 1
How to Install Hadoop on Ubuntu: A Step-by-Step Guide
#
hadoop
#
hdfs
#
ubuntu
#
bigdata
Comments
Add Comment
10 min read
🤔 Is It Possible to Achieve 100% Test Automation?
Carlos Gonzaga
Carlos Gonzaga
Carlos Gonzaga
Follow
Jul 18
🤔 Is It Possible to Achieve 100% Test Automation?
#
softwarequality
#
softwaredevelopment
#
automatedtesting
#
bigdata
Comments
Add Comment
2 min read
Data ingestion – definition, types and best practices
DBSync
DBSync
DBSync
Follow
Jul 23
Data ingestion – definition, types and best practices
#
cloud
#
data
#
bigdata
Comments
Add Comment
8 min read
How to Handle Databases with Billions of Records
DbVisualizer
DbVisualizer
DbVisualizer
Follow
Aug 12
How to Handle Databases with Billions of Records
#
bigdata
2
reactions
Comments
Add Comment
1 min read
Effective Strategies for Scaling Databases: Enhancing Performance for Growing Data Needs
Nguyen Gia Huy
Nguyen Gia Huy
Nguyen Gia Huy
Follow
Aug 6
Effective Strategies for Scaling Databases: Enhancing Performance for Growing Data Needs
#
database
#
learning
#
webdev
#
bigdata
4
reactions
Comments
Add Comment
5 min read
Databricks - Variant Type Analysis
Debashis Adak
Debashis Adak
Debashis Adak
Follow
Jun 29
Databricks - Variant Type Analysis
#
databricks
#
spark
#
bigdata
#
datalake
Comments
Add Comment
7 min read
Working with Parquet files in Java using Carpet
Jerónimo López
Jerónimo López
Jerónimo López
Follow
Jun 19
Working with Parquet files in Java using Carpet
#
parquet
#
java
#
bigdata
#
dataengineering
1
reaction
Comments
Add Comment
6 min read
Optimizing ETL Processes for Efficient Data Loading in EDWs
Ovais
Ovais
Ovais
Follow
Jul 12
Optimizing ETL Processes for Efficient Data Loading in EDWs
#
emterprisedatawarehouse
#
etl
#
datascience
#
bigdata
Comments
Add Comment
4 min read
Patient-Centered Care and Data Integration in Population Health Management
Ovais
Ovais
Ovais
Follow
Jul 12
Patient-Centered Care and Data Integration in Population Health Management
#
powerapps
#
healthcare
#
datascience
#
bigdata
Comments
Add Comment
4 min read
The Basics of Big Data: What You Need to Know
bvanderbilt0033
bvanderbilt0033
bvanderbilt0033
Follow
Jun 7
The Basics of Big Data: What You Need to Know
#
dataprotection
#
dataanalytics
#
dataprivacy
#
bigdata
Comments
Add Comment
3 min read
loading...
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account