Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to print definition of a function in Scala

I was wondering if we can print the definition of a function in Scala. A function is treated as an object in Scala.

For example:

scala> val splitFunction = (value : String) => { value.split(" ")}

splitFunction: String => Array[String] = <function1>

Above, Scala interactive shell indicates that splitFunction has input parameter String and it returns Array of strings. What really function1 indicates here?

Is it possible to print or retrieve the definition of splitFunction?

We can achieve same in python: Is it possible to print a function as a string in Python?

Update: In Apache Spark, RDD lineage or DAG stores information about parent RDD and transformation at each stage. I am interested in fetching definition of a function (even lambda or anonymous functions) used as an argument to transformations such as flatMap or map.

For example: File - DebugTest.scala

val dataRDD = sc.textFile( "README.md" )
val splitFunction = (value : String) => {value.split(" ")}
val mapRDD = dataRDD.map(splitFunction )

println(mapRDD.toDebugString)

Output:

(1) MapPartitionsRDD[2] at map at DebugTest.scala:43 []
 |  README.md MapPartitionsRDD[1] at textFile at DebugTest.scala:41 []
 |  README.md HadoopRDD[0] at textFile at DebugTest.scala:41 []

From above output, I can understand what transformations are performed but cannot understand or retrieve definition for the splitFunction used as an argument in "map" transformation. Is there any way to retrieve or print it?

like image 546
aagora Avatar asked Oct 10 '17 22:10

aagora


1 Answers

No. (In general)

The reason it says <function1> is because there is no good string representation of a function to be given, so we just say that it's some function that takes one argument.

The reason you can't get the definition of a function is because Scala is compiled. The JVM bytecode for your function is already rather unreadable (comments mine, of course)

aload_1           // Load the second argument (of reference type) onto the stack (the first is this function object)
checkcast #mm     // Where mm is an index into the class constant pool, representing a reference to the class String (cast inserted because this is a generic method)
ldc #nn           // Where nn is an index representing the string " "
invokevirtual #oo // Where oo is another index, this time representing String::split
areturn           // Return whatever's left on the stack

A "show function implementation" function would have to 1) retrieve the implementation from the JVM (I believe there's an API for that, but it's meant for debuggers), and then 2) decompile the code to Scala/Java. It's not impossible, per se, but I don't think anyone's done it (and really, why would you do it?)

Now, it would be possible for every Scala anonymous function to just store its code and override toString to output it, but, again, there's simply no reason to do it. Even if you want the implementation for debugging purposes, chances are you have the source code and you can use the line numbers in class file to jump to it, and if you want to store it, it's already stored in the class file.

If you really want it, it is possible, in theory, to define a (opt-in) macro (or even a compiler plugin)

import language.experimental.macros
import reflect.macros.whitebox.Context
def stringF(f: Any): Any = macro stringF_impl
def stringF_impl(c: whitebox.Context)(f: c.Tree): c.Tree = ???

That turns

stringF { arg => body } // BTW: say bye to type inference

into

new Function1[A, R] {
  override def apply(arg: A) = body
  override def toString() = "{ arg => body }" // This part is not there normally
}

But, again, I haven't heard of anyone doing it, and there's just no strong reason to try.

like image 98
HTNW Avatar answered Dec 25 '22 07:12

HTNW