I know that you can mark a scala object as @serializable
, but I don't understand what to do with the object afterwards. Do I simply treat it as a Java Serializable object?
I want to serialize the object into a stream of bytes. Can someone show me the code to transform a serialize object into either a byte array or a string?
(the googles have not been helpful with this question)
FOLLOWUP: Thanks. I now understand that I can use it like a Java Serializable
object. Sometimes the obvious answer escapes me.
To make a Scala class serializable, extend the Serializable trait and add the @SerialVersionUID annotation to the class: @SerialVersionUID ( 100L ) class Stock ( var symbol : String , var price : BigDecimal ) extends Serializable { // code here ... }
Serializing an object means taking the data stored in an object and converting it to bytes (or a string). Suppose you want to write the data in an object to a JSON file. JSON files store strings. JSON has no understanding about the JVM or Scala objects.
In Scala, an object is a named instance with members such as fields and methods. An object and a class that have the same name and which are defined in the same source file are known as companions. Companions has special access control properties, which is covered under Scala/Access modifiers.
Java serialization: By default, Spark serializes objects using Java's ObjectOutputStream framework, and can work with any class you create that implements java. io. Serializable . You can also control the performance of your serialization more closely by extending java.
To answer your first question: yes you can treat it as a Java Serializable object:
scala> @serializable object A
defined module A
scala> import java.io.ByteArrayOutputStream;
import java.io.ByteArrayOutputStream
scala> import java.io.ObjectOutputStream;
import java.io.ObjectOutputStream
scala> val baos = new ByteArrayOutputStream(1024)
baos: java.io.ByteArrayOutputStream =
scala> val o = new ObjectOutputStream(baos)
o: java.io.ObjectOutputStream = java.io.ObjectOutputStream@3d689405
scala> o.writeObject(A)
scala> baos.toByteArray
res4: Array[Byte] = Array(-84, -19, 0, 5, 115, 114, 0, 24, 108, 105, 110, 101, 49, 51, 36, 111, 98, 106, 101, 99, 116, 36, 36, 105, 119, 36, 36, 105, 119, 36, 65, 36, 110, -104, -28, -53, -123, -97, -118, -36, 2, 0, 0, 120, 112)
scala> object B
defined module B
scala> o.writeObject(B)
java.io.NotSerializableException: B$
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1156)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)
at .(:13)
at .()
at RequestResult$.(:9)
at RequestResult$.()
at RequestResult$scala_repl_result()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at scala.tools.nsc.Interpreter$Request$$anonfun$loadAndRun$1$$anonfun$apply$18.apply(Interpreter.scala:981)
at scala.tools.nsc.Interpreter$Request$...
If you want to serialize to some string format, perhaps this library may be useful, which serializes scala objects into JSON.
In general, I wouldn't recommend serializing Objects, however it could be a way to send your "global state" across the net.
As for how to send/receive data check out java.io.ObjectOutputStream and java.io.ObjectInputStream.
scala> trait Test { def x : Int }
defined trait Test
scala> @serializable object Foo { var x = 5 }
defined module Foo
scala> import java.io._
import java.io._
scala> def write() {
| val output = new ObjectOutputStream(new FileOutputStream("test.obj"))
| output.writeObject(Foo)
| output.close()
| }
write: ()Unit
scala> write()
scala> def read() = {
| val input = new ObjectInputStream(new FileInputStream("test.obj"))
| val obj = input.readObject()
| input.close()
| obj
| }
read: ()java.lang.Object
scala> Foo.x = 7
scala> val r = read()
r: Test = Foo$@2855e552
scala> r.x
res39: Int = 7
You see... top level objects don't really work well with serialization. However a nested object could be serialized. For example:
scala> @serializable
| class SomeClass(var y : Int) {
| @serializable object X extends Test { def x = y }
| }
defined class SomeClass
scala> def write(x : AnyRef) {
| val output = new ObjectOutputStream(new FileOutputStream("test.obj"))
| output.writeObject(x)
| output.close()
| }
write: (x : AnyRef)Unit
scala> def read[A] = {
| val input = new ObjectInputStream(new FileInputStream("test.obj"))
| val obj = input.readObject()
| input.close()
| obj.asInstanceOf[A]
| }
read: [A]A
scala> write(x.X)
scala> val y = read[Test]
y: Test = SomeClass$X$@58e39f23
scala> y.x
res51: Int = 10
scala> x.y = 20
scala> x.X.x
res52: Int = 20
scala> y.x
res53: Int = 10
Hope that helps!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With