Performance Impact of RDD to JavaRDD conversion

Question

I have a code something like this and I want to work on JavaRDD instead of RDD. So, I'm doing conversion here. I would like to know the performance impact of this transformation specially when I'm dealing with GBs of data.

RDD<String> textFile = sc.textFile(filePath, 2);
JavaRDD<String> javaRDD = textFile.toJavaRDD(); 

Is this wide transformation or narrow ? What is the difference between JavaRDD and RDD ?

Answer 1

There's no significant performance penalty - JavaRDD is a simple wrapper around RDD just to make calls from Java code more convenient. It holds the original RDD as its member, and calls that member's method on any method invocation, for example (from JavaRDD.scala):

def cache(): JavaRDD[T] = wrapRDD(rdd.cache()) 

wrapRDD boils down to something like new JavaRDD[T](rdd), so the only performance penalty is creating a thin Java object for every method invocation, but that's entirely negligible as it's not done per element in the RDD, but once for the entire object.