Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I know spark-core version?

I am using spark 1.5.2 in hdp, and version for hadoop is 2.7.1.2.3.4.7-4. When I attempt to add jars in maven pom file like this

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

I dont know where to find version for spark-core. There are 2.11 and 2.10. any help is appreciated.

like image 484
Frankie Avatar asked Aug 05 '16 17:08

Frankie


1 Answers

That version you are mentioning denotes which version of Scala you want to use for the spark-core.

You need to check Scala's version on your cluster to know if it's 2.10 that you need or 2.11.

like image 75
eliasah Avatar answered Sep 21 '22 17:09

eliasah