Apache Spark Mlib

Question

I'm trying to use the MLlib from Spark to implement KMeans on Java and I've stumbled upon a problem, which is that, despite the fact that I've imported the correct jar's my compiler will not recognise this line

// Cluster the data into two classes using KMeans
    int numClusters = 2;
    int numIterations = 20;
    KMeansModel clusters = KMeans.train(parsedData.rdd(), numClusters, numIterations);

The error I get is: The method train(<RDD> vector, int, int) is undefined for the type KMeans() Which doesn't make any sense since, I've downloaded the latest apache MLlib(1.5.2) jar and also it is defined in the Javadoc.

Any ideas? Has anyone encountered this sort of problem before?

Answer 1

I was getting the similar issue and it was resolved by importing the correct libraries,

import org.apache.spark.mllib.clustering.KMeans;
import org.apache.spark.mllib.clustering.KMeansModel;

instead of

import org.apache.spark.ml.clustering.KMeans;
import org.apache.spark.ml.clustering.KMeansModel;