The SPARK_HOME env variable is set but Jupyter Notebook doesn't see it. (Windows)

Question

I'm on Windows 10. I was trying to get Spark up and running in a Jupyter Notebook alongside Python 3.5. I installed a pre-built version of Spark and set the SPARK_HOME environmental variable. I installed findspark and run the code:

import findspark
findspark.init()

I receive a Value error:

ValueError: Couldn't find Spark, make sure SPARK_HOME env is set or Spark is in an expected location (e.g. from homebrew installation).

However the SPARK_HOME variable is set. Here is a screenshot that shows that the list of environmental variables on my system.

Has anyone encountered this issue or would know how to fix this? I only found an old discussion in which someone had set SPARK_HOME to the wrong folder but I don't think it's my case.

Answer 1

I had the same problem and wasted a lot of time. I found two solutions:

There are two solutions

1. copy downloaded spark folder in somewhere in C directory and give the link as below

   import findspark
   findspark.init('C:/spark')
   

2. use the function of findspark to find automatically the spark folder

   import findspark
   findspark.find()