I installed spark on my mac with homebrew. I'm trying to find the directory where I installed it. I've tried googling it but I'm not having much luck. It doesn't seem like it should be that tricky. Can anyone please tell me what I need to run in mac terminal or from spark shell to find the installation directory for spark?
Update:
Code:
brew info apache-spark
Output:
apache-spark: stable 2.3.2, HEAD
Engine for large-scale data processing
https://spark.apache.org/
/usr/local/Cellar/apache-spark/2.3.2 (1,058 files, 244.6MB) *
Built from source on 2018-10-30 at 14:16:30
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-spark.rb
==> Requirements
Required: java = 1.8 ✔
==> Options
--HEAD
Install HEAD version
==> Analytics
install: 4,534 (30 days), 14,340 (90 days), 56,698 (365 days)
install_on_request: 4,263 (30 days), 13,490 (90 days), 51,876 (365 days)
build_error: 0 (30 days)
code:
which spark-shell
Output:
/Users/sshields/anaconda2/bin/spark-shell
We have successfully installed Apache Spark on Mac OS. The installation directory would be /usr/local/Cellar/apache-spark/.
On Intel Macs, Homebrew, and any packages you install using Homebrew, go in /usr/local/bin . Homebrew chose /usr/local/bin because it is already in your PATH by default.
1. Spark Version Check from Command Line. Like any other tools or language, you can use –version option with spark-submit , spark-shell , and spark-sql to find the version. All above spark-submit command, spark-shell command, and spark-sql return the below output where you can find Spark installed version.
You should use brew info apache-spark
and it will include the path in the output if you brew install
it (I did not so it's not in the output below)
$ brew info apache-spark
apache-spark: stable 2.3.2, HEAD
Engine for large-scale data processing
https://spark.apache.org/
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-spark.rb
==> Requirements
Required: java = 1.8 ✔
==> Options
--HEAD
Install HEAD version
==> Analytics
install: 4,534 (30 days), 14,340 (90 days), 56,698 (365 days)
install_on_request: 4,263 (30 days), 13,490 (90 days), 51,876 (365 days)
build_error: 0 (30 days)
From the website:
Homebrew installs packages to their own directory and then symlinks their files into /usr/local.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With