Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Log in
Create account
DEV Community
Close
#
spark
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Data storage patterns, versioning and partitions
Karun Japhet
Karun Japhet
Karun Japhet
Follow
May 9 '21
Data storage patterns, versioning and partitions
#
datascience
#
bigdata
#
spark
#
s3
11
reactions
Comments
Add Comment
9 min read
Apache Spark and BigQuery with AWS Sagemaker Studio
Ramon Marrero
Ramon Marrero
Ramon Marrero
Follow
for
AWS Community Builders
Jun 14 '21
Apache Spark and BigQuery with AWS Sagemaker Studio
#
sagemaker
#
amazonwebservices
#
aws
#
spark
Comments
Add Comment
1 min read
My Journey With Spark On Kubernetes... In Python (1/3)
Pascal Gillet
Pascal Gillet
Pascal Gillet
Follow
for
Stack Labs
Apr 12 '21
My Journey With Spark On Kubernetes... In Python (1/3)
#
spark
#
kubernetes
#
python
49
reactions
Comments
Add Comment
9 min read
My Journey With Spark On Kubernetes... In Python (2/3)
Pascal Gillet
Pascal Gillet
Pascal Gillet
Follow
for
Stack Labs
Apr 12 '21
My Journey With Spark On Kubernetes... In Python (2/3)
#
spark
#
kubernetes
#
python
23
reactions
Comments
Add Comment
9 min read
My Journey With Spark On Kubernetes... In Python (3/3)
Pascal Gillet
Pascal Gillet
Pascal Gillet
Follow
for
Stack Labs
Apr 12 '21
My Journey With Spark On Kubernetes... In Python (3/3)
#
spark
#
kubernetes
#
python
20
reactions
Comments
1
comment
17 min read
Unit testing your PySpark library
Darren Fuller
Darren Fuller
Darren Fuller
Follow
Mar 28 '21
Unit testing your PySpark library
#
python
#
spark
#
testing
#
pyspark
9
reactions
Comments
Add Comment
9 min read
How to recover from a deleted _spark_metadata folder in Spark Structured Streaming
Kevin Wallimann
Kevin Wallimann
Kevin Wallimann
Follow
Mar 11 '21
How to recover from a deleted _spark_metadata folder in Spark Structured Streaming
#
spark
10
reactions
Comments
3
comments
5 min read
Spark and Docker: Your Spark development cycle just got 10x faster !
JY @ DataMechanics
JY @ DataMechanics
JY @ DataMechanics
Follow
Nov 23 '20
Spark and Docker: Your Spark development cycle just got 10x faster !
#
spark
#
docker
#
kubernetes
#
devops
15
reactions
Comments
Add Comment
7 min read
How-to guide: Set up, Manage & Monitor Spark on Kubernetes
JY @ DataMechanics
JY @ DataMechanics
JY @ DataMechanics
Follow
Nov 20 '20
How-to guide: Set up, Manage & Monitor Spark on Kubernetes
#
spark
#
kubernetes
#
docker
#
cloudnative
20
reactions
Comments
Add Comment
10 min read
Apache Spark Java Tutorial: Simplest Guide to Get Started
hellocodeclub
hellocodeclub
hellocodeclub
Follow
Nov 9 '20
Apache Spark Java Tutorial: Simplest Guide to Get Started
#
machinelearning
#
spark
#
java
#
bigdata
10
reactions
Comments
Add Comment
3 min read
Is Structured Streaming Exactly-Once? Well, it depends...
Kevin Wallimann
Kevin Wallimann
Kevin Wallimann
Follow
Nov 6 '20
Is Structured Streaming Exactly-Once? Well, it depends...
#
spark
10
reactions
Comments
Add Comment
4 min read
can a map function be executed on multiple executors for an item in RDD.
Sachin Gupta
Sachin Gupta
Sachin Gupta
Follow
Oct 2 '20
can a map function be executed on multiple executors for an item in RDD.
#
spark
3
reactions
Comments
Add Comment
1 min read
Predicting machine failures with distributed computing (Spark, AWS EMR, and DL)
Musa Atlıhan
Musa Atlıhan
Musa Atlıhan
Follow
Oct 26 '20
Predicting machine failures with distributed computing (Spark, AWS EMR, and DL)
#
spark
#
emr
#
bigdata
#
deeplearning
9
reactions
Comments
Add Comment
10 min read
Using Aerospike Connect For Spark
Ken Tune
Ken Tune
Ken Tune
Follow
for
Aerospike
Oct 21 '20
Using Aerospike Connect For Spark
#
aerospike
#
spark
6
reactions
Comments
Add Comment
5 min read
Migrating from a plain Spark Application to ZIO with ZparkIO
Ayoub Fakir
Ayoub Fakir
Ayoub Fakir
Follow
Oct 16 '20
Migrating from a plain Spark Application to ZIO with ZparkIO
#
scala
#
spark
#
zio
#
functional
9
reactions
Comments
Add Comment
6 min read
Spark: unit, integration and end-to-end tests.
Gustavo Martin Morcuende
Gustavo Martin Morcuende
Gustavo Martin Morcuende
Follow
for
Adevinta Spain
Oct 15 '20
Spark: unit, integration and end-to-end tests.
#
scala
#
spark
#
testing
20
reactions
Comments
Add Comment
5 min read
Spark Journey begins...
MoRoth
MoRoth
MoRoth
Follow
Sep 28 '20
Spark Journey begins...
#
spark
#
python
#
bigdata
8
reactions
Comments
Add Comment
3 min read
Working with nested structures in Spark
Saša Zejnilović
Saša Zejnilović
Saša Zejnilović
Follow
Sep 20 '20
Working with nested structures in Spark
#
spark
#
bigdata
#
scala
#
library
7
reactions
Comments
1
comment
3 min read
Intoduction to Apache Spark
maninekkalapudi
maninekkalapudi
maninekkalapudi
Follow
Sep 14 '20
Intoduction to Apache Spark
#
dataengineering
#
apachespark
#
bigdata
#
spark
10
reactions
Comments
Add Comment
6 min read
Spark Side Menu Micro-Interactions Deconstruction
Dusko Stamenic
Dusko Stamenic
Dusko Stamenic
Follow
Aug 29 '20
Spark Side Menu Micro-Interactions Deconstruction
#
microinteraction
#
email
#
spark
#
frontend
3
reactions
Comments
Add Comment
2 min read
Unit Testing Apache Spark Structured Streaming using MemoryStream
Bartosz Gajda
Bartosz Gajda
Bartosz Gajda
Follow
Aug 10 '20
Unit Testing Apache Spark Structured Streaming using MemoryStream
#
spark
#
apachespark
#
testing
#
bigdata
7
reactions
Comments
Add Comment
4 min read
Setting up IntelliJ IDEA for Apache Spark and Scala development
Bartosz Gajda
Bartosz Gajda
Bartosz Gajda
Follow
Aug 10 '20
Setting up IntelliJ IDEA for Apache Spark and Scala development
#
spark
#
scala
#
intellij
#
tutorial
6
reactions
Comments
Add Comment
2 min read
Exploiting Schema Inference in Apache Spark
Bartosz Gajda
Bartosz Gajda
Bartosz Gajda
Follow
Aug 10 '20
Exploiting Schema Inference in Apache Spark
#
spark
#
scala
#
bigdata
#
datascience
2
reactions
Comments
Add Comment
3 min read
How to create a low-cost Apache Spark cluster on Microsoft Azure
Sascha Dittmann
Sascha Dittmann
Sascha Dittmann
Follow
Jul 18 '20
How to create a low-cost Apache Spark cluster on Microsoft Azure
#
bigdata
#
azure
#
spark
#
azurebatch
7
reactions
Comments
Add Comment
4 min read
How to make a column non-nullable in Spark Structured Streaming
Kevin Wallimann
Kevin Wallimann
Kevin Wallimann
Follow
Jul 11 '20
How to make a column non-nullable in Spark Structured Streaming
#
spark
4
reactions
Comments
Add Comment
2 min read
loading...
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account