Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does a method parameter cause NotSerializableException with Mockito?

I have some Scala code which runs with spark, but I simplified them:

// Not Serializable
class Config

object FileReader extends FileReader
class FileReader extends Serializable {
  def read(config: Config): String = config.getClass.toString
}

object Task extends Task(FileReader)
class Task(fileReader: FileReader) extends Serializable {
  def execute(config: Config): Unit = {
    fileReader.read(config)
  }
}

Not the Config is not Serializable.

I want to write some tests for them, and the instance of Task need to be Serializable because it may be serialized and send to spark workers.

I use this function to check if an object can be serialized or not:

def checkSerializable(obj: AnyRef, name: String) = {
  println("### checking " + name + ": " + obj.getClass)
  new ObjectOutputStream(new ByteArrayOutputStream()).writeObject(obj)
  println(name + " ok")
}

First let's check if a normal Task instance can be serialized:

val task = new Task(new FileReader)
checkSerializable(task, "no-mockito")

Outputs:

### checking no-mockito: class Task
no-mockito ok

Seems OK.

But I want to mock the FileReader with Mockito, so my code will be:

val fileReader = Mockito.mock(classOf[FileReader])
val config = Mockito.mock(classOf[Config])
Mockito.when(fileReader.read(config)).thenReturn("mocked")
val task = new Task(fileReader)
checkSerializable(task, "with-mockito1")

It reports error that Config is not serializable:

### checking with-mockito1: class Task
java.io.NotSerializableException: Config$$EnhancerByMockitoWithCGLIB$$c7dcb0a5
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1165)
    at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1359)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1155)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1535)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)

This is strange because config is just some method parameters, not class fields!

I modified my code a little bit to make the config serializable:

val fileReader = Mockito.mock(classOf[FileReader])
val config = Mockito.mock(classOf[Config], Mockito.withSettings().serializable())
Mockito.when(fileReader.read(config)).thenReturn("mocked")
val task = new Task(fileReader)
checkSerializable(task, "with-mockito2")

It still fails, with another NotSerializableException:

### checking with-mockito2: class Task
java.io.NotSerializableException: org.mockito.internal.creation.DelegatingMethod
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1165)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1535)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1413)

Not sure why config instance will be included in task when working with Mockito? And how to avoid it?

Demo project: https://github.com/freewind/mockito-serialization-issue, you can just clone it and run demo/MockitoDemo.scala


Update:

Another notable thing is: if I remove this line

fileReader.read(config)

from Task, that means the Task will be:

class Task(fileReader: FileReader) extends Serializable {
  def execute(config: Config): Unit = {
    // removed this line: fileReader.read(config)
  }
}

no NotSerializableException will be thrown anymore (I didn't change the testing code)

like image 746
Freewind Avatar asked Sep 03 '15 15:09

Freewind


People also ask

What are the causes of NotSerializableException in spark?

The error was triggered when I initialize a variable on the driver (master), but then tried to use it on one of the workers. When that happens, Spark Streaming will try to serialize the object to send it over to the worker, and fail if the object is not serializable. I solved the error by making the variable static.

Why is Java Optional not Serializable?

This answer is in response to the question in the title, "Shouldn't Optional be Serializable?" The short answer is that the Java Lambda (JSR-335) expert group considered and rejected it.

What is Serializable in Scala?

Serializing an object means taking the data stored in an object and converting it to bytes (or a string). Suppose you want to write the data in an object to a JSON file. JSON files store strings. JSON has no understanding about the JVM or Scala objects.


1 Answers

You probably will need to make the FileReader mock serializable too try:

val fileReader = Mockito.mock(classOf[FileReader], Mockito.withSettings().serializable())
like image 184
Mike Pone Avatar answered Sep 24 '22 20:09

Mike Pone