Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does persist() on spark by default store to memory or disk?

I'm aware that, on spark, you can change persist() to store data either to memory or disk, but I was wondering what the default is. I've tried searching for this but haven't been able to get a clear answer to this.

like image 867
LentilSoup Avatar asked Jan 31 '26 21:01

LentilSoup


1 Answers

The default storage level of persist is MEMORY_ONLY you can find details from here. The other option can be MEMORY_AND_DISK, MEMORY_ONLY_SER , MEMORY_AND_DISK_SERMEMORY_ONLY_2, MEMORY_AND_DISK_2, DISK_ONLY, OFF_HEAP (experimental).

Here is an simple explanation to help you.

like image 73
koiralo Avatar answered Feb 04 '26 00:02

koiralo



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!