Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

system cannot find the path specified in spark-shell

Tags:

apache-spark

I am trying to install Spark in Win7 and am getting an error.

These are my environment settings:

SPARK_HOME : C:\spark (this is where I have unzipped the spark files)

JAVA_HOME : C:\Program Files\Java\jdk1.7.0_71;

SCALA_HOME: C:\Program Files (x86)\scala

PATH : C:\Program Files\Java\jdk1.7.0_71\bin;C:\app\Sampad\product\11.2.0\dbhome_1\bin;C:\Python27;C:\Program Files\Python27\;C:\Program Files\Python27\Scripts;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Java\jdk1.6.0_45\bin;C:\Program Files\nodejs\;C:\Program Files\Python27;C:\Anaconda;C:\Anaconda\Scripts;C:\HashiCorp\Vagrant\bin;C:\Program Files (x86)\scala\bin;C:\spark\bin;

HADOOP_HOME : C:\winutils; (set this after reading this blog

Please let me know what mistake I have made. Thanks in advance.

like image 542
user1548787 Avatar asked Nov 16 '15 11:11

user1548787


2 Answers

I installed java and spark in folder with out spaces and I am using windows 10.

In my case I added

JAVA_HOME=C:\Java\jdk1.8.0_191\bin

so it was trying to search for executables in "C:\Program Files\Java\jdk1.8.0_191\bin\bin"

make sure to add variable as

JAVA_HOME = C:\Java\jdk1.8.0_191

and In path environmental variable add

%JAVA_HOME%\bin

""same for SPARK_HOME""

It is working for me now !!!

like image 178
Hamsavardhini Avatar answered Nov 24 '22 00:11

Hamsavardhini


Probably, it happens, because different java versions:

JAVA_HOME : C:\Program Files\Java\jdk1.7.0_71;

C:\Program Files\Java\jdk1.6.0_45\bin

Instead of "C:\Program Files\Java\jdk1.6.0_45\bin" use "%JAVA_HOME%\bin"

Spark: Trying to run spark-shell, but get 'cmd' is not recognized as an internal or

like image 21
constructor Avatar answered Nov 24 '22 00:11

constructor