Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reuse a cached Spark RDD

Is there a possibility in Spark to re-use a cached RDD in another application (or in another run of the same application)?

JavaRDD<ExampleClass> toCache = ... // transformations on the RDD
toCache.cache();                    // can this be reused somehow in another application or further runs?
like image 728
D. Müller Avatar asked Nov 30 '25 07:11

D. Müller


1 Answers

No, Spark RDD cannot be used in other application or in another run.

You can connect Spark with for example Hazelcast or Apache Ignite to save RDDs in memory. Other application will have possibility to read data saved in first application

like image 197
T. Gawęda Avatar answered Dec 06 '25 00:12

T. Gawęda



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!