DEV Community

Cover image for How to install Apache PySpark on Mac using Python?
Ruthvik Raja M.V
Ruthvik Raja M.V

Posted on

How to install Apache PySpark on Mac using Python?

Hello,

Apache PySpark works with Java 8 version and not with the latest Java version so, make sure that you install the correct version to run Apache PySpark on your Machine.

Download Java 8 from the following link and install the software:
https://www.java.com/en/download/manual.jsp

If you already have the latest Java version on your Machine and want to remove the latest Java software from your Machine, then please visit the following blog:

https://dev.to/ruthvikraja_mv/how-to-uninstall-java-on-mac-104a

Now, its time to check the installed Java version on your Mac, enter the following command in the terminal to check the version:

java -version
Enter fullscreen mode Exit fullscreen mode

Launch Anaconda Navigator (or) any other IDE to run python code for installing Apache PySpark on your Machine. Type the following command and hit enter:

pip install pyspark
Enter fullscreen mode Exit fullscreen mode

It will take some time to install the software and once the software got installed you can check the version by entering the following command:

pyspark
Enter fullscreen mode Exit fullscreen mode

Once everything is done please type the following command in the terminal to check whether we can create a new session using PySpark:

spark-shell
Enter fullscreen mode Exit fullscreen mode

If the software has been successfully installed on your machine without any dependency errors then it should show as follows:

Image description

Done...

Top comments (0)