What happens if an RDD can't fit into memory in Spark? [duplicate]

Question

As far as I know, Spark tries to do all computation in memory, unless you call persist with disk storage option. If however, we don't use any persist, what does Spark do when an RDD doesn't fit in memory? What if we have very huge data. How will Spark handle it without crashing?

Answer 1

From Apache Spark FAQ's:

Spark's operators spill data to disk if it does not fit in memory, allowing it to run well on any sized data. Likewise, cached datasets that do not fit in memory are either spilled to disk or recomputed on the fly when needed, as determined by the RDD's storage level.

Refer below link to know more about storage levels and how to choose appropriate one between these levels: programming-guide.html