how to properly build spark 2.0 from source, to include pyspark?

Question

I've just built spark 2.0 on Ubuntu host, using "sbt assembly". Everything finished fine, but, when I attempted to submit a pyspark job:

bin/spark-submit --master spark://localhost:7077 examples/src/main/python/pi.py 1000

I got this error:

Failed to find Spark jars directory (/home/ubuntu/spark/spark-2.0.0/assembly/target/scala-2.10/jars).
You need to build Spark with the target "package" before running this program.

What should I do in order to re-build spark 2.0 to include pyspark?

Answer 1

Try:

1. Install sbt

2. Build:

   https://github.com/apache/spark.git
   cd spark
   git checkout v2.0.0
   sbt package