I am using spark 1.5.2 in hdp, and version for hadoop is 2.7.1.2.3.4.7-4. When I attempt to add jars in maven pom file like this
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.5.2</version>
</dependency>
I dont know where to find version for spark-core. There are 2.11 and 2.10. any help is appreciated.
That version you are mentioning denotes which version of Scala you want to use for the spark-core.
You need to check Scala's version on your cluster to know if it's 2.10 that you need or 2.11.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With