DEV Community

# bigdata

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Reliable ingestion from AWS S3 using Hudi

Reliable ingestion from AWS S3 using Hudi

3
Comments
6 min read
ทดสอบการทำ Anonymize data in your data lake with Amazon Athena

ทดสอบการทำ Anonymize data in your data lake with Amazon Athena

9
Comments 1
2 min read
เริ่มใช้งาน SQL-based INSERTS, DELETES and UPSERTS in S3 โดยใช้ AWS Glue 3.0 และ Delta Lake

เริ่มใช้งาน SQL-based INSERTS, DELETES and UPSERTS in S3 โดยใช้ AWS Glue 3.0 และ Delta Lake

11
Comments
6 min read
How to deal with Big data challenges

How to deal with Big data challenges

6
Comments
5 min read
SQL-based INSERTS, DELETES and UPSERTS in S3 using AWS Glue 3.0 and Delta Lake

SQL-based INSERTS, DELETES and UPSERTS in S3 using AWS Glue 3.0 and Delta Lake

21
Comments 8
8 min read
To let the beginners know their career goals who have opted data science.

To let the beginners know their career goals who have opted data science.

2
Comments
2 min read
Updating data files, commits vs. pull requests

Updating data files, commits vs. pull requests

6
Comments 4
3 min read
Unboxing a Database-How Databases Work Internally

Unboxing a Database-How Databases Work Internally

28
Comments 4
11 min read
Data Optimization for Compacted Partitions

Data Optimization for Compacted Partitions

3
Comments
8 min read
Apache Hudi - The Streaming Data Lake Platform

Apache Hudi - The Streaming Data Lake Platform

3
Comments
25 min read
UPSERTS and DELETES using AWS Glue and Delta Lake

UPSERTS and DELETES using AWS Glue and Delta Lake

26
Comments 4
10 min read
Exploratory Data Analysis Using Python

Exploratory Data Analysis Using Python

45
Comments 1
5 min read
Getting started with Azure Data Explorer and Azure Synapse Analytics for Big Data processing

Getting started with Azure Data Explorer and Azure Synapse Analytics for Big Data processing

4
Comments
9 min read
E-commerce Security Basics: How to Start with E-commerce Security

E-commerce Security Basics: How to Start with E-commerce Security

2
Comments
6 min read
Creating a Spark Standalone Cluster with Docker and docker-compose(2021 update)

Creating a Spark Standalone Cluster with Docker and docker-compose(2021 update)

49
Comments 4
7 min read
Understanding Open Telemetry and Observability w/ Splunk's Spiros Xanthos

Understanding Open Telemetry and Observability w/ Splunk's Spiros Xanthos

12
Comments
1 min read
How to Use Consistent Hashing in a System Design Interview?

How to Use Consistent Hashing in a System Design Interview?

10
Comments 3
7 min read
The Complete Guide to Data Science, Big Data, and Data Analytics

The Complete Guide to Data Science, Big Data, and Data Analytics

11
Comments
3 min read
How to easily install kafka without zookeeper

How to easily install kafka without zookeeper

5
Comments
7 min read
AWS Data Lake with Terraform - Part 2 of 6

AWS Data Lake with Terraform - Part 2 of 6

22
Comments
2 min read
AWS Data Lake with Terraform - Part 1 of 6

AWS Data Lake with Terraform - Part 1 of 6

29
Comments
4 min read
Assess how many Kafka servers are needed to face a scenario of 1 billion requests.

Assess how many Kafka servers are needed to face a scenario of 1 billion requests.

7
Comments
6 min read
Big Data + MySQL = Mission InnoPossible?

Big Data + MySQL = Mission InnoPossible?

4
Comments
9 min read
A Visual Guide To: Azure Data Factory

A Visual Guide To: Azure Data Factory

12
Comments
4 min read
AzureFunBytes Reminder - Intro to @Azure Data Factory with @KromerBigData - 5/13/2021

AzureFunBytes Reminder - Intro to @Azure Data Factory with @KromerBigData - 5/13/2021

7
Comments
3 min read
5 Ways Big Data & Analytics Can Pay Off To Your Marketing & Sales in 2021

5 Ways Big Data & Analytics Can Pay Off To Your Marketing & Sales in 2021

5
Comments 2
6 min read
AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData

AzureFunBytes Episode 43 - Intro to @Azure Data Factory with @KromerBigData

6
Comments
3 min read
Eliminate frictions from the developers’ experience – discover the new Inspector data visualization UI

Eliminate frictions from the developers’ experience – discover the new Inspector data visualization UI

4
Comments
3 min read
Data storage patterns, versioning and partitions

Data storage patterns, versioning and partitions

11
Comments
9 min read
Here is What Happens If You Decouple Your BI Stack

Here is What Happens If You Decouple Your BI Stack

5
Comments
7 min read
The New & Improved Spark UI & Spark History Server is now Generally Available

The New & Improved Spark UI & Spark History Server is now Generally Available

3
Comments
9 min read
Efficient Iteration of Big Data in Django

Efficient Iteration of Big Data in Django

5
Comments 1
4 min read
On explaining technical stuff in a non-technical way — (Py)Spark

On explaining technical stuff in a non-technical way — (Py)Spark

4
Comments
7 min read
ALGORITHMIC TRADING

ALGORITHMIC TRADING

6
Comments
3 min read
Event Streaming and AWS Kinesis

Event Streaming and AWS Kinesis

22
Comments
4 min read
Introduction to Data Analysis and Visualization using Python

Introduction to Data Analysis and Visualization using Python

4
Comments
4 min read
Introduction to Apache Airflow: get started in 5 minutes

Introduction to Apache Airflow: get started in 5 minutes

11
Comments
8 min read
Data, data everywhere!

Data, data everywhere!

5
Comments
2 min read
Inside Presto Optimizer

Inside Presto Optimizer

7
Comments
9 min read
Best Online Courses for Data Engineers In 2021

Best Online Courses for Data Engineers In 2021

33
Comments
7 min read
3 CTOs And Founders Perspectives on the Modern Data Stack

3 CTOs And Founders Perspectives on the Modern Data Stack

5
Comments
11 min read
How I Built a Data Discovery API for AWS Data Lake

How I Built a Data Discovery API for AWS Data Lake

4
Comments
7 min read
"I learned right away how important Data manipulation and cleaning was to managing business affairs", — Matthew D. Groves

"I learned right away how important Data manipulation and cleaning was to managing business affairs", — Matthew D. Groves

6
Comments
4 min read
"Working with Data helps to uncover the inner workings of multiple spheres in our daily life",— Roksolana Diachuk.

"Working with Data helps to uncover the inner workings of multiple spheres in our daily life",— Roksolana Diachuk.

3
Comments
4 min read
"Data is always something new to learn in the area, some new tool to try, some new insight to discover", — Ruben Berenguel

"Data is always something new to learn in the area, some new tool to try, some new insight to discover", — Ruben Berenguel

2
Comments
4 min read
"Data is the new center of gravity", — Jules Damji.

"Data is the new center of gravity", — Jules Damji.

2
Comments
3 min read
The Management of Data

The Management of Data

8
Comments 1
3 min read
Configuration of Hadoop Cluster Using Ansible

Configuration of Hadoop Cluster Using Ansible

2
Comments
4 min read
How we implemented Distributed Multi-document ACID Transactions in Couchbase

How we implemented Distributed Multi-document ACID Transactions in Couchbase

7
Comments
14 min read
7 Real-Time Data Streaming Tools You Should Consider On Your Next Project

7 Real-Time Data Streaming Tools You Should Consider On Your Next Project

21
Comments 1
9 min read
Business Analytics tools & use cases

Business Analytics tools & use cases

4
Comments
7 min read
Elasticsearch as a primary database?

Elasticsearch as a primary database?

20
Comments
2 min read
Why You Need a CRM Data Cleanup

Why You Need a CRM Data Cleanup

2
Comments
8 min read
Big Data, What's the Big Deal

Big Data, What's the Big Deal

1
Comments
2 min read
Starting your Journey with Big Data Analytics

Starting your Journey with Big Data Analytics

37
Comments
4 min read
Impact of COVID-19 on people's habits worldwide

Impact of COVID-19 on people's habits worldwide

8
Comments 5
2 min read
Data Engineering skills

Data Engineering skills

13
Comments 1
3 min read
Apache Kafka: What is and how it works

Apache Kafka: What is and how it works

6
Comments 1
8 min read
Getting Started With JanusGraph

Getting Started With JanusGraph

2
Comments
5 min read
What is data engineering?

What is data engineering?

4
Comments
1 min read
loading...