I'm aware that, on spark, you can change persist() to store data either to memory or disk, but I was wondering what the default is. I've tried searching for this but haven't been able to get a clear answer to this.
The default storage level of persist is MEMORY_ONLY you can find details from here. The other option can be MEMORY_AND_DISK, MEMORY_ONLY_SER , MEMORY_AND_DISK_SERMEMORY_ONLY_2, MEMORY_AND_DISK_2, DISK_ONLY, OFF_HEAP (experimental).
Here is an simple explanation to help you.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With