Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why do you extend Serializable in Scala?

I'm going through some Sapark projects in Scala, and I see that all of the objects are extending Serializable .

It's like :

object someName extends Serializable {
//some code
}

I understand that serialization is often used for storing or communicating data structures, so that the data can be easily loaded into memory in the original form, from the serialized form. However, in this case the object is more like a Java class. So, what's the point or advantage of extending Serializable ? When do you do this? Is it necessary to always do this?

like image 773
div93 Avatar asked Apr 17 '19 05:04

div93


People also ask

Why do we need serialization in spark?

Serialization is used for performance tuning on Apache Spark. All data that is sent over the network or written to the disk or persisted in the memory should be serialized. Serialization plays an important role in costly operations.

What is the main purpose of serialization?

Serialization is the process of converting an object into a stream of bytes to store the object or transmit it to memory, a database, or a file. Its main purpose is to save the state of an object in order to be able to recreate it when needed.

What does serializable mean in Scala?

Serializing an object means taking the data stored in an object and converting it to bytes (or a string). Suppose you want to write the data in an object to a JSON file. JSON files store strings. JSON has no understanding about the JVM or Scala objects.

What happens if we don't use serializable?

What happens if you try to send non-serialized Object over network? When traversing a graph, an object may be encountered that does not support the Serializable interface. In this case the NotSerializableException will be thrown and will identify the class of the non-serializable object.


1 Answers

You only actually need it when objects are used other than calling methods directly on them. E.g. something like

val x = someName

rdd.map { ... x ... }

or

object SomeName extends Serializable, SomeTrait { ... }

def f(x: SomeTrait) = ...

rdd.map { ... f(SomeName) ... }

For "global" objects not extending any classes/traits it's basically useless because no one ever does what the first snippet shows, but it doesn't hurt either. Note that Nil and None are examples of the second and they do extend Serializable (even without Spark).

like image 65
Alexey Romanov Avatar answered Oct 13 '22 18:10

Alexey Romanov