Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Search
Log in
Create account
DEV Community
Close
#
bigdata
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Reliable ingestion from AWS S3 using Hudi
vinoth chandar
vinoth chandar
vinoth chandar
Follow
Sep 2 '21
Reliable ingestion from AWS S3 using Hudi
#
datascience
#
bigdata
#
analytics
#
aws
3
reactions
Comments
Add Comment
6 min read
ทดสอบการทำ Anonymize data in your data lake with Amazon Athena
Chatchai Komrangded (Bas)
Chatchai Komrangded (Bas)
Chatchai Komrangded (Bas)
Follow
for
AWS Community ASEAN
Aug 31 '21
ทดสอบการทำ Anonymize data in your data lake with Amazon Athena
#
aws
#
awsthai
#
bigdata
#
datascience
9
reactions
Comments
1
comment
2 min read
เริ่มใช้งาน SQL-based INSERTS, DELETES and UPSERTS in S3 โดยใช้ AWS Glue 3.0 และ Delta Lake
Chatchai Komrangded (Bas)
Chatchai Komrangded (Bas)
Chatchai Komrangded (Bas)
Follow
for
AWS Community ASEAN
Aug 30 '21
เริ่มใช้งาน SQL-based INSERTS, DELETES and UPSERTS in S3 โดยใช้ AWS Glue 3.0 และ Delta Lake
#
aws
#
awsthai
#
bigdata
#
datascience
11
reactions
Comments
Add Comment
6 min read
How to deal with Big data challenges
Adamo Software
Adamo Software
Adamo Software
Follow
Aug 30 '21
How to deal with Big data challenges
#
challenge
#
bigdata
#
news
6
reactions
Comments
Add Comment
5 min read
SQL-based INSERTS, DELETES and UPSERTS in S3 using AWS Glue 3.0 and Delta Lake
Kyle Escosia
Kyle Escosia
Kyle Escosia
Follow
for
AWS Community ASEAN
Aug 23 '21
SQL-based INSERTS, DELETES and UPSERTS in S3 using AWS Glue 3.0 and Delta Lake
#
aws
#
tutorial
#
bigdata
#
datascience
21
reactions
Comments
8
comments
8 min read
To let the beginners know their career goals who have opted data science.
Devansh Tayal
Devansh Tayal
Devansh Tayal
Follow
Aug 18 '21
To let the beginners know their career goals who have opted data science.
#
datascience
#
machinelearning
#
bigdata
#
cloud
2
reactions
Comments
Add Comment
2 min read
Updating data files, commits vs. pull requests
Nicolas Fränkel
Nicolas Fränkel
Nicolas Fränkel
Follow
Aug 15 '21
Updating data files, commits vs. pull requests
#
github
#
bigdata
#
automation
#
git
6
reactions
Comments
4
comments
3 min read
Unboxing a Database-How Databases Work Internally
Elegberun Olugbenga
Elegberun Olugbenga
Elegberun Olugbenga
Follow
Jul 30 '21
Unboxing a Database-How Databases Work Internally
#
database
#
bigdata
#
distributedsystems
28
reactions
Comments
4
comments
11 min read
Data Optimization for Compacted Partitions
Dustin Smith
Dustin Smith
Dustin Smith
Follow
Jul 28 '21
Data Optimization for Compacted Partitions
#
bigdata
#
datascience
#
spark
#
dataplatforms
3
reactions
Comments
Add Comment
8 min read
Apache Hudi - The Streaming Data Lake Platform
vinoth chandar
vinoth chandar
vinoth chandar
Follow
Jul 27 '21
Apache Hudi - The Streaming Data Lake Platform
#
datascience
#
analytics
#
bigdata
#
database
3
reactions
Comments
Add Comment
25 min read
UPSERTS and DELETES using AWS Glue and Delta Lake
Kyle Escosia
Kyle Escosia
Kyle Escosia
Follow
for
AWS Community ASEAN
Jul 21 '21
UPSERTS and DELETES using AWS Glue and Delta Lake
#
aws
#
tutorial
#
bigdata
#
analytics
26
reactions
Comments
4
comments
10 min read
Exploratory Data Analysis Using Python
Mwenda Harun Mbaabu
Mwenda Harun Mbaabu
Mwenda Harun Mbaabu
Follow
Jul 17 '21
Exploratory Data Analysis Using Python
#
python
#
datascience
#
machinelearning
#
bigdata
45
reactions
Comments
1
comment
5 min read
Getting started with Azure Data Explorer and Azure Synapse Analytics for Big Data processing
Abhishek Gupta
Abhishek Gupta
Abhishek Gupta
Follow
for
Microsoft Azure
Jul 17 '21
Getting started with Azure Data Explorer and Azure Synapse Analytics for Big Data processing
#
bigdata
#
analytics
#
python
#
programming
4
reactions
Comments
Add Comment
9 min read
E-commerce Security Basics: How to Start with E-commerce Security
Jhon Wilson
Jhon Wilson
Jhon Wilson
Follow
Jun 30 '21
E-commerce Security Basics: How to Start with E-commerce Security
#
security
#
beginners
#
bigdata
2
reactions
Comments
Add Comment
6 min read
Creating a Spark Standalone Cluster with Docker and docker-compose(2021 update)
Marco Villarreal
Marco Villarreal
Marco Villarreal
Follow
Jun 27 '21
Creating a Spark Standalone Cluster with Docker and docker-compose(2021 update)
#
docker
#
spark
#
bigdata
49
reactions
Comments
4
comments
7 min read
Understanding Open Telemetry and Observability w/ Splunk's Spiros Xanthos
Conor Bronsdon
Conor Bronsdon
Conor Bronsdon
Follow
for
LinearB
Jun 23 '21
Understanding Open Telemetry and Observability w/ Splunk's Spiros Xanthos
#
opentelemetry
#
observability
#
bigdata
#
datascience
12
reactions
Comments
Add Comment
1 min read
How to Use Consistent Hashing in a System Design Interview?
Arslan Ahmad
Arslan Ahmad
Arslan Ahmad
Follow
Jun 19 '21
How to Use Consistent Hashing in a System Design Interview?
#
distributedsystems
#
bigdata
#
career
#
architecture
10
reactions
Comments
3
comments
7 min read
The Complete Guide to Data Science, Big Data, and Data Analytics
Le Truong
Le Truong
Le Truong
Follow
Jun 11 '21
The Complete Guide to Data Science, Big Data, and Data Analytics
#
datascience
#
bigdata
11
reactions
Comments
Add Comment
3 min read
How to easily install kafka without zookeeper
Aditya Sridhar
Aditya Sridhar
Aditya Sridhar
Follow
Jun 7 '21
How to easily install kafka without zookeeper
#
kafka
#
tutorial
#
beginners
#
bigdata
5
reactions
Comments
Add Comment
7 min read
AWS Data Lake with Terraform - Part 2 of 6
Augusto Valdivia
Augusto Valdivia
Augusto Valdivia
Follow
for
AWS Community Builders
Jun 3 '21
AWS Data Lake with Terraform - Part 2 of 6
#
aws
#
terraform
#
awsdatalake
#
bigdata
22
reactions
Comments
Add Comment
2 min read
AWS Data Lake with Terraform - Part 1 of 6
Augusto Valdivia
Augusto Valdivia
Augusto Valdivia
Follow
for
AWS Community Builders
Jun 1 '21
AWS Data Lake with Terraform - Part 1 of 6
#
aws
#
terraform
#
awsdatalake
#
bigdata
29
reactions
Comments
Add Comment
4 min read
Assess how many Kafka servers are needed to face a scenario of 1 billion requests.
fante-sun
fante-sun
fante-sun
Follow
May 27 '21
Assess how many Kafka servers are needed to face a scenario of 1 billion requests.
#
kafka
#
architecture
#
bigdata
7
reactions
Comments
Add Comment
6 min read
Big Data + MySQL = Mission InnoPossible?
Arctype Team
Arctype Team
Arctype Team
Follow
for
Arctype
May 25 '21
Big Data + MySQL = Mission InnoPossible?
#
database
#
mysql
#
bigdata
#
innodb
4
reactions
Comments
Add Comment
9 min read
A Visual Guide To: Azure Data Factory
Nitya Narasimhan, Ph.D
Nitya Narasimhan, Ph.D
Nitya Narasimhan, Ph.D
Follow
for
Microsoft Azure
May 24 '21
A Visual Guide To: Azure Data Factory
#
azure
#
sketchnote
#
bigdata
#
beginners
12
reactions
Comments
Add Comment
4 min read
AzureFunBytes Reminder - Intro to @Azure Data Factory with @KromerBigData - 5/13/2021
Jay Gordon
Jay Gordon
Jay Gordon
Follow
for
Microsoft Azure
May 12 '21
AzureFunBytes Reminder - Intro to @Azure Data Factory with @KromerBigData - 5/13/2021
#
bigdata
#
azure
#
tutorial
#
beginners
7
reactions
Comments
Add Comment
3 min read
5 Ways Big Data & Analytics Can Pay Off To Your Marketing & Sales in 2021
Shifa Martin
Shifa Martin
Shifa Martin
Follow
May 18 '21
5 Ways Big Data & Analytics Can Pay Off To Your Marketing & Sales in 2021
#
bigdata
#
analytics
#
development
#
solutions
5
reactions
Comments
2
comments
6 min read
AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData
Jay Gordon
Jay Gordon
Jay Gordon
Follow
for
Microsoft Azure
May 13 '21
AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData
#
azure
#
bigdata
#
etl
#
beginners
6
reactions
Comments
Add Comment
3 min read
Eliminate frictions from the developers’ experience – discover the new Inspector data visualization UI
Valerio
Valerio
Valerio
Follow
for
Inspector.dev
May 11 '21
Eliminate frictions from the developers’ experience – discover the new Inspector data visualization UI
#
ui
#
ux
#
bigdata
#
monitoring
4
reactions
Comments
Add Comment
3 min read
Data storage patterns, versioning and partitions
Karun Japhet
Karun Japhet
Karun Japhet
Follow
May 9 '21
Data storage patterns, versioning and partitions
#
datascience
#
bigdata
#
spark
#
s3
11
reactions
Comments
Add Comment
9 min read
Here is What Happens If You Decouple Your BI Stack
Anna Geller
Anna Geller
Anna Geller
Follow
May 7 '21
Here is What Happens If You Decouple Your BI Stack
#
analytics
#
bigdata
#
microservices
#
architecture
5
reactions
Comments
Add Comment
7 min read
The New & Improved Spark UI & Spark History Server is now Generally Available
JY @ DataMechanics
JY @ DataMechanics
JY @ DataMechanics
Follow
May 7 '21
The New & Improved Spark UI & Spark History Server is now Generally Available
#
apachespark
#
sparkui
#
bigdata
#
monitoring
3
reactions
Comments
Add Comment
9 min read
Efficient Iteration of Big Data in Django
Kyle Johnson
Kyle Johnson
Kyle Johnson
Follow
May 6 '21
Efficient Iteration of Big Data in Django
#
django
#
bigdata
#
python
5
reactions
Comments
1
comment
4 min read
On explaining technical stuff in a non-technical way — (Py)Spark
Maria Karanasou
Maria Karanasou
Maria Karanasou
Follow
Apr 23 '21
On explaining technical stuff in a non-technical way — (Py)Spark
#
python
#
programming
#
distributedsystems
#
bigdata
4
reactions
Comments
Add Comment
7 min read
ALGORITHMIC TRADING
Praveen Reddy Pingala
Praveen Reddy Pingala
Praveen Reddy Pingala
Follow
Apr 27 '21
ALGORITHMIC TRADING
#
algorithms
#
datascience
#
bigdata
6
reactions
Comments
Add Comment
3 min read
Event Streaming and AWS Kinesis
Supratip Banerjee
Supratip Banerjee
Supratip Banerjee
Follow
for
AWS Community Builders
Apr 26 '21
Event Streaming and AWS Kinesis
#
aws
#
bigdata
#
eventdriven
#
architecture
22
reactions
Comments
Add Comment
4 min read
Introduction to Data Analysis and Visualization using Python
Tanay Js
Tanay Js
Tanay Js
Follow
Apr 26 '21
Introduction to Data Analysis and Visualization using Python
#
datascience
#
bigdata
#
python
#
database
4
reactions
Comments
Add Comment
4 min read
Introduction to Apache Airflow: get started in 5 minutes
Erin Schaffer
Erin Schaffer
Erin Schaffer
Follow
for
Educative
Apr 22 '21
Introduction to Apache Airflow: get started in 5 minutes
#
bigdata
#
datascience
#
programming
11
reactions
Comments
Add Comment
8 min read
Data, data everywhere!
Ebraim Carvalho
Ebraim Carvalho
Ebraim Carvalho
Follow
Apr 21 '21
Data, data everywhere!
#
database
#
datascience
#
bigdata
5
reactions
Comments
Add Comment
2 min read
Inside Presto Optimizer
Vladimir Ozerov
Vladimir Ozerov
Vladimir Ozerov
Follow
Apr 20 '21
Inside Presto Optimizer
#
presto
#
sql
#
database
#
bigdata
7
reactions
Comments
Add Comment
9 min read
Best Online Courses for Data Engineers In 2021
SeattleDataGuy
SeattleDataGuy
SeattleDataGuy
Follow
Apr 18 '21
Best Online Courses for Data Engineers In 2021
#
database
#
datascience
#
career
#
bigdata
33
reactions
Comments
Add Comment
7 min read
3 CTOs And Founders Perspectives on the Modern Data Stack
SeattleDataGuy
SeattleDataGuy
SeattleDataGuy
Follow
Apr 16 '21
3 CTOs And Founders Perspectives on the Modern Data Stack
#
database
#
analytics
#
datascience
#
bigdata
5
reactions
Comments
Add Comment
11 min read
How I Built a Data Discovery API for AWS Data Lake
Taavi Rehemägi
Taavi Rehemägi
Taavi Rehemägi
Follow
for
Dashbird
Apr 13 '21
How I Built a Data Discovery API for AWS Data Lake
#
aws
#
serverless
#
bigdata
#
api
4
reactions
Comments
Add Comment
7 min read
"I learned right away how important Data manipulation and cleaning was to managing business affairs", — Matthew D. Groves
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Follow
for
Konfy
Apr 8 '21
"I learned right away how important Data manipulation and cleaning was to managing business affairs", — Matthew D. Groves
#
datascience
#
nosql
#
distributedsystems
#
bigdata
6
reactions
Comments
Add Comment
4 min read
"Working with Data helps to uncover the inner workings of multiple spheres in our daily life",— Roksolana Diachuk.
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Follow
for
Konfy
Apr 8 '21
"Working with Data helps to uncover the inner workings of multiple spheres in our daily life",— Roksolana Diachuk.
#
bigdata
#
kubernetes
#
datascience
3
reactions
Comments
Add Comment
4 min read
"Data is always something new to learn in the area, some new tool to try, some new insight to discover", — Ruben Berenguel
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Follow
for
Konfy
Apr 8 '21
"Data is always something new to learn in the area, some new tool to try, some new insight to discover", — Ruben Berenguel
#
bigdata
#
datascience
2
reactions
Comments
Add Comment
4 min read
"Data is the new center of gravity", — Jules Damji.
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Anastasia Khomyakova ❤
Follow
for
Konfy
Apr 8 '21
"Data is the new center of gravity", — Jules Damji.
#
datascience
#
bigdata
2
reactions
Comments
Add Comment
3 min read
The Management of Data
Anamika
Anamika
Anamika
Follow
Apr 6 '21
The Management of Data
#
database
#
systems
#
bigdata
#
management
8
reactions
Comments
1
comment
3 min read
Configuration of Hadoop Cluster Using Ansible
Piyush Bagani
Piyush Bagani
Piyush Bagani
Follow
Mar 27 '21
Configuration of Hadoop Cluster Using Ansible
#
bigdata
#
ansible
#
aws
#
arth
2
reactions
Comments
Add Comment
4 min read
How we implemented Distributed Multi-document ACID Transactions in Couchbase
deniswsrosa
deniswsrosa
deniswsrosa
Follow
Mar 23 '21
How we implemented Distributed Multi-document ACID Transactions in Couchbase
#
database
#
bigdata
7
reactions
Comments
Add Comment
14 min read
7 Real-Time Data Streaming Tools You Should Consider On Your Next Project
SeattleDataGuy
SeattleDataGuy
SeattleDataGuy
Follow
Mar 21 '21
7 Real-Time Data Streaming Tools You Should Consider On Your Next Project
#
database
#
datascience
#
bigdata
#
analytics
21
reactions
Comments
1
comment
9 min read
Business Analytics tools & use cases
Apiumhub
Apiumhub
Apiumhub
Follow
Mar 25 '21
Business Analytics tools & use cases
#
technologyindustrytr
#
bigdata
4
reactions
Comments
Add Comment
7 min read
Elasticsearch as a primary database?
Internet Explorer
Internet Explorer
Internet Explorer
Follow
Mar 16 '21
Elasticsearch as a primary database?
#
elasticsearch
#
database
#
bigdata
#
datascience
20
reactions
Comments
Add Comment
2 min read
Why You Need a CRM Data Cleanup
Abe Dearmer
Abe Dearmer
Abe Dearmer
Follow
for
Xplenty inc.
Feb 9 '21
Why You Need a CRM Data Cleanup
#
database
#
datascience
#
bigdata
2
reactions
Comments
Add Comment
8 min read
Big Data, What's the Big Deal
Soumitra Banerjee
Soumitra Banerjee
Soumitra Banerjee
Follow
Mar 13 '21
Big Data, What's the Big Deal
#
datascience
#
bigdata
1
reaction
Comments
Add Comment
2 min read
Starting your Journey with Big Data Analytics
Adit Modi
Adit Modi
Adit Modi
Follow
for
Cloud Tech
Mar 10 '21
Starting your Journey with Big Data Analytics
#
aws
#
bigdata
#
career
#
beginners
37
reactions
Comments
Add Comment
4 min read
Impact of COVID-19 on people's habits worldwide
Igor Lukanin
Igor Lukanin
Igor Lukanin
Follow
Mar 4 '21
Impact of COVID-19 on people's habits worldwide
#
showdev
#
webdev
#
javascript
#
bigdata
8
reactions
Comments
5
comments
2 min read
Data Engineering skills
luminousmen
luminousmen
luminousmen
Follow
Mar 14 '21
Data Engineering skills
#
bigdata
#
bi
#
mlops
#
data
13
reactions
Comments
1
comment
3 min read
Apache Kafka: What is and how it works
joaosczip
joaosczip
joaosczip
Follow
Mar 2 '21
Apache Kafka: What is and how it works
#
microservices
#
programming
#
bigdata
6
reactions
Comments
1
comment
8 min read
Getting Started With JanusGraph
Sunny Srinidhi
Sunny Srinidhi
Sunny Srinidhi
Follow
Feb 25 '21
Getting Started With JanusGraph
#
janusgraph
#
bigdata
#
graph
#
datascience
2
reactions
Comments
Add Comment
5 min read
What is data engineering?
Nočnica Mellifera
Nočnica Mellifera
Nočnica Mellifera
Follow
for
RudderStack
Feb 19 '21
What is data engineering?
#
data
#
bigdata
#
engineering
#
rudderstack
4
reactions
Comments
Add Comment
1 min read
loading...
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account