Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I use a @serializable scala object?

I know that you can mark a scala object as @serializable, but I don't understand what to do with the object afterwards. Do I simply treat it as a Java Serializable object?

I want to serialize the object into a stream of bytes. Can someone show me the code to transform a serialize object into either a byte array or a string?

(the googles have not been helpful with this question)

FOLLOWUP: Thanks. I now understand that I can use it like a Java Serializable object. Sometimes the obvious answer escapes me.

like image 860
Fred Haslam Avatar asked Aug 09 '10 16:08

Fred Haslam


People also ask

How do you serialize an object in Scala?

To make a Scala class serializable, extend the Serializable trait and add the @SerialVersionUID annotation to the class: @SerialVersionUID ( 100L ) class Stock ( var symbol : String , var price : BigDecimal ) extends Serializable { // code here ... }

What is serializable object in Scala?

Serializing an object means taking the data stored in an object and converting it to bytes (or a string). Suppose you want to write the data in an object to a JSON file. JSON files store strings. JSON has no understanding about the JVM or Scala objects.

What is Scala object?

In Scala, an object is a named instance with members such as fields and methods. An object and a class that have the same name and which are defined in the same source file are known as companions. Companions has special access control properties, which is covered under Scala/Access modifiers.

How do you do serialization in spark?

Java serialization: By default, Spark serializes objects using Java's ObjectOutputStream framework, and can work with any class you create that implements java. io. Serializable . You can also control the performance of your serialization more closely by extending java.


2 Answers

To answer your first question: yes you can treat it as a Java Serializable object:


scala> @serializable object A
defined module A

scala> import java.io.ByteArrayOutputStream;
import java.io.ByteArrayOutputStream

scala> import java.io.ObjectOutputStream;
import java.io.ObjectOutputStream

scala> val baos = new ByteArrayOutputStream(1024)
baos: java.io.ByteArrayOutputStream = 

scala> val o = new ObjectOutputStream(baos)
o: java.io.ObjectOutputStream = java.io.ObjectOutputStream@3d689405

scala> o.writeObject(A)

scala> baos.toByteArray
res4: Array[Byte] = Array(-84, -19, 0, 5, 115, 114, 0, 24, 108, 105, 110, 101, 49, 51, 36, 111, 98, 106, 101, 99, 116, 36, 36, 105, 119, 36, 36, 105, 119, 36, 65, 36, 110, -104, -28, -53, -123, -97, -118, -36, 2, 0, 0, 120, 112)

scala> object B
defined module B

scala> o.writeObject(B)
java.io.NotSerializableException: B$
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
    at .(:13)
    at .()
    at RequestResult$.(:9)
    at RequestResult$.()
    at RequestResult$scala_repl_result()
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at scala.tools.nsc.Interpreter$Request$$anonfun$loadAndRun$1$$anonfun$apply$18.apply(Interpreter.scala:981)
    at scala.tools.nsc.Interpreter$Request$...

If you want to serialize to some string format, perhaps this library may be useful, which serializes scala objects into JSON.

like image 123
Arjan Blokzijl Avatar answered Oct 06 '22 00:10

Arjan Blokzijl


In general, I wouldn't recommend serializing Objects, however it could be a way to send your "global state" across the net.

As for how to send/receive data check out java.io.ObjectOutputStream and java.io.ObjectInputStream.

scala> trait Test { def x : Int }
defined trait Test

scala> @serializable object Foo { var x = 5 }
defined module Foo


scala> import java.io._                                                     
import java.io._

scala> def write() {
     | val output = new ObjectOutputStream(new FileOutputStream("test.obj"))
     | output.writeObject(Foo)
     | output.close()
     | }
write: ()Unit


scala> write()

scala> def read() = {
     | val input = new ObjectInputStream(new FileInputStream("test.obj"))
     | val obj = input.readObject()
     | input.close()
     | obj
     | }
read: ()java.lang.Object

scala> Foo.x = 7

scala> val r = read()
r: Test = Foo$@2855e552

scala> r.x
res39: Int = 7

You see... top level objects don't really work well with serialization. However a nested object could be serialized. For example:

scala> @serializable
     | class SomeClass(var y : Int) {
     |   @serializable object X extends Test {  def x  = y }
     | }
defined class SomeClass


scala> def write(x : AnyRef) {
     | val output = new ObjectOutputStream(new FileOutputStream("test.obj"))
     | output.writeObject(x)
     | output.close()
     | }
write: (x : AnyRef)Unit

scala> def read[A] = {
     | val input = new ObjectInputStream(new FileInputStream("test.obj"))
     | val obj = input.readObject()
     | input.close()
     | obj.asInstanceOf[A]
     | }
read: [A]A

scala> write(x.X)

scala> val y = read[Test]
y: Test = SomeClass$X$@58e39f23

scala> y.x
res51: Int = 10

scala> x.y = 20  

scala> x.X.x
res52: Int = 20

scala> y.x
res53: Int = 10

Hope that helps!

like image 36
jsuereth Avatar answered Oct 06 '22 00:10

jsuereth