DEV Community

Greeshma Pogula
Greeshma Pogula

Posted on

Load Data

Read CSV file:
df = spark.read.format("CSV").option("header","true").load(input_path)

df = df.na.drop(how="any")

To save as output files

data.coalesce(1).write.mode("overwrite").csv(output_path,header=True)

Spark by examples

Top comments (0)