I need to install spark and run it in standalone mode on one machine and looking for a straight forward way to install it via apt-get
I found how to do this with pyspark via pip here
I can not find any way to install spark with one terminal command.
Does installing pyspark install all the spark software?
I found instructions for how to install spark, and it's more complex
Is there a way to install spark with a similar one terminal command via apt-get?
EDIT
I found this explaining how to install spark using apt-get and i get the following error
E: Unable to locate package spark-core E: Unable to locate package spark-master E: Unable to locate package spark-worker E: Unable to locate package spark-history-server E: Unable to locate package spark-python
Thanks
Before installing pyspark you must install Java8. This is described at
For a fully automatic installation of Java8 on Ubuntu do
$ apt-get update
$ apt-get -y install software-properties-common
$ add-apt-repository -y ppa:webupd8team/java
$ echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
$ echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections
$ apt-get update
$ apt-get -y install oracle-java8-installer
(see https://newfivefour.com/docker-java8-auto-install.html)
Afterwards you can simply run pip install pyspark
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With