Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to find installation directory of Apache Spark package in Homebrew?

I installed spark on my mac with homebrew. I'm trying to find the directory where I installed it. I've tried googling it but I'm not having much luck. It doesn't seem like it should be that tricky. Can anyone please tell me what I need to run in mac terminal or from spark shell to find the installation directory for spark?

Update:

Code:

brew info apache-spark

Output:

apache-spark: stable 2.3.2, HEAD
Engine for large-scale data processing
https://spark.apache.org/
/usr/local/Cellar/apache-spark/2.3.2 (1,058 files, 244.6MB) *
  Built from source on 2018-10-30 at 14:16:30
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-spark.rb
==> Requirements
Required: java = 1.8 ✔
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 4,534 (30 days), 14,340 (90 days), 56,698 (365 days)
install_on_request: 4,263 (30 days), 13,490 (90 days), 51,876 (365 days)
build_error: 0 (30 days)

code:

which spark-shell

Output:

/Users/sshields/anaconda2/bin/spark-shell
like image 510
user3476463 Avatar asked Nov 19 '18 19:11

user3476463


People also ask

Where is Apache Spark installed on Mac?

We have successfully installed Apache Spark on Mac OS. The installation directory would be /usr/local/Cellar/apache-spark/.

Where does brew install packages on Mac?

On Intel Macs, Homebrew, and any packages you install using Homebrew, go in /usr/local/bin . Homebrew chose /usr/local/bin because it is already in your PATH by default.

How do I know if spark is installed on Mac?

1. Spark Version Check from Command Line. Like any other tools or language, you can use –version option with spark-submit , spark-shell , and spark-sql to find the version. All above spark-submit command, spark-shell command, and spark-sql return the below output where you can find Spark installed version.


1 Answers

You should use brew info apache-spark and it will include the path in the output if you brew install it (I did not so it's not in the output below)

$ brew info apache-spark
apache-spark: stable 2.3.2, HEAD
Engine for large-scale data processing
https://spark.apache.org/
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-spark.rb
==> Requirements
Required: java = 1.8 ✔
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 4,534 (30 days), 14,340 (90 days), 56,698 (365 days)
install_on_request: 4,263 (30 days), 13,490 (90 days), 51,876 (365 days)
build_error: 0 (30 days)

From the website:

Homebrew installs packages to their own directory and then symlinks their files into /usr/local.

like image 168
Jacek Laskowski Avatar answered Nov 15 '22 05:11

Jacek Laskowski