💥Spark Partitions Shuffle

The Spark SQL shuffle is a mechanism for redistributing or re-partitioning data so that the data is grouped differently across partitions

Episode source