Isn't lambda function also an object with Function1 trait?

Question

object MyApp {
  def printValues(f: {def apply(x: Int): Int}, from: Int, to: Int): Unit = {
    println(
      (from to to).map(f(_)).mkString(" ")
    )
  }

  def main(args: Array[String]): Unit = {
    val anonfun1 = new Function1[Int, Int] {
      final def apply(x: Int): Int = x * x
    }

    val fun1 = (x:Int)=>x*x
    printValues(fun1, 3, 6)
  }
}

I thought that lambda function in scala is also an object that extends the Function1 trait. However this code fails for printValues(fun1, 3, 6) and not printlnValues(anonfun1, 3, 6). Why is that so?

Answer 1

This was a very interesting question to explore. There is a saying that one should never rely on implementation detail in code, I think this is borderline of doing that.

Let's try and break down what happens here.

Structural Types:

When you require a structural type, such as you did in this method:

def printValues(f: {def apply(x: Int): Int}, from: Int, to: Int): Unit = {
  println(
    (from to to).map(f(_)).mkString(" ")
  )
}

What Scala does is use reflection to try and find the apply method at run-time, and dynamically invoke it. It gets translated to something that looks like this:

public static Method reflMethod$Method1(final Class x$1) {
    MethodCache methodCache1 = Tests$$anonfun$printValues$1.reflPoly$Cache1.get();
    if (methodCache1 == null) {
        methodCache1 = (MethodCache)new EmptyMethodCache();
        Tests$$anonfun$printValues$1.reflPoly$Cache1 = new SoftReference((T)methodCache1);
    }
    Method method1 = methodCache1.find(x$1);
    if (method1 != null) {
        return method1;
    }
    method1 = ScalaRunTime$.MODULE$.ensureAccessible(x$1.getMethod("apply", (Class[])Tests$$anonfun$printValues$1.reflParams$Cache1));
    Tests$$anonfun$printValues$1.reflPoly$Cache1 = new SoftReference((T)methodCache1.add(x$1, method1));
    return method1;
}

This is the decompiled Java code that gets emitted. Long story short, it looks for the apply method.

What happens in Scala <= 2.11

For any Scala version prior 2.12, declaring an anonymous function results in the compiler generating a class extending AbstractFunction*, where * is the arity of the function. These abstract function classes inherit in turn Function*, and implement their apply method with the implementation of the lambda.

So for example, if we take your expression:

val fun1 = (x:Int) => x * x 

The compiler emits for us:

val fun2: Int => Int = {
    @SerialVersionUID(value = 0) final <synthetic> class $anonfun extends scala.runtime.AbstractFunction1$mcII$sp with Serializable {
      def <init>(): <$anon: Int => Int> = {
        $anonfun.super.<init>();
        ()
      };
      final def apply(x: Int): Int = $anonfun.this.apply$mcII$sp(x);
      <specialized> def apply$mcII$sp(x: Int): Int = x.*(x)
    };
    (new <$anon: Int => Int>(): Int => Int)
  };
  ()

When we look at the bytecode level, we see the generated anonymous class and it's apply methods:

Compiled from "Tests.scala"
public final class othertests.Tests$$anonfun$1 extends scala.runtime.AbstractFunction1$mcII$sp implements scala.Serializable {
  public static final long serialVersionUID;

  public final int apply(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokevirtual #21                 // Method apply$mcII$sp:(I)I
       5: ireturn

  public int apply$mcII$sp(int);
    Code:
       0: iload_1
       1: iload_1
       2: imul
       3: ireturn

So what happens when you request the `apply` method at runtime? The run-time will see that theirs a method defined on `$anonfun` called `apply` which takes an `Int` and returns an `Int`, which is exactly what we want and invoke it. All is good and everyone's happy.

Lo and Behold, Scala 2.12

In Scala 2.12, we get something called SAM conversion. SAM types is a feature in Java 8 which allows you to abbreviate implementing an interface, and instead supply a lambda expression. For example:

new Thread(() -> System.out.println("Yay in lambda!")).start();

Instead of having to implement Runnable and override public void run. Scala 2.12 set a goal to be compatible with SAM types via SAM conversion when possible.

In our particular case, a SAM conversion is possible, meaning that instead of Scala's Function1[Int, Int] we get a specialized version of scala.runtime.java8.JFunction1$mcII$sp. This JFunction is compatible with Java, and has the following structure:

package scala.runtime.java8;

@FunctionalInterface
public interface JFunction1$mcII$sp extends scala.Function1, java.io.Serializable {
    int apply$mcII$sp(int v1);

    default Object apply(Object t) { return scala.runtime.BoxesRunTime.boxToInteger(apply$mcII$sp(scala.runtime.BoxesRunTime.unboxToInt(t))); }
}

This JFunction1 has been specialized (like we use the @specialized annotation in Scala) to emit a special method for def apply(i: Int): Int. Note one important factor here, that this method only implements an apply method of the form Object => Object, not Int => Int. Now we can start getting a feeling as to why this could be problematic.

Now, when we compile the same example in Scala 2.12, we see:

def main(args: Array[String]): Unit = {
  val fun2: Int => Int = {
    final <artifact> def $anonfun$main(x: Int): Int = x.*(x);
    ((x: Int) => $anonfun$main(x))
  };
  ()

We no longer see a method extending AbstractFunction*, we simply see method called $anonfun$main. When we look at the generated bytecode, we see that internally, it'll call JFunction1$mcII$sp.apply$mcII$sp(int v1);:

public void main(java.lang.String[]);
  Code:
    0: invokedynamic #41,  0  // InvokeDynamic #0:apply$mcII$sp: ()Lscala/runtime/java8/JFunction1$mcII$sp;
    5: astore_2
    6: return

Yet if we explicitly extends Function1 ourselves and implement apply, we get the a similar behavior to previous Scala version, but not an exact identical:

def main(args: Array[String]): Unit = {
  val anonfun1: Int => Int = {
    final class $anon extends AnyRef with Int => Int {
      def <init>(): <$anon: Int => Int> = {
        $anon.super.<init>();
        ()
      };
      final def apply(x: Int): Int = x.*(x)
    };
    new $anon()
  };
  {
    ()
  }
}

We no longer extend AbstractFunction*, but we do have an apply method which satisfies the structural type condition during run-time. At the bytecode level, we see an int apply(int), an object apply(object) and a bunch of cases for the @specialization attribute annotating Function*:

public final int apply(int);
  Code:
     0: aload_0
     1: iload_1
     2: invokevirtual #183                // Method apply$mcII$sp:(I)I
     5: ireturn

public int apply$mcII$sp(int);
  Code:
     0: iload_1
     1: iload_1
     2: imul
     3: ireturn

public final java.lang.Object apply(java.lang.Object);
  Code:
     0: aload_0
     1: aload_1
     2: invokestatic  #190                // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
     5: invokevirtual #192                // Method apply:(I)I
     8: invokestatic  #196                // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
    11: areturn

Conclusion:

We can see that there's a change in the implementation detail of how the Scala compiler treats lambda expressions under some circumstances. Is this a bug? my feelings tends towards a no. In no where does the Scala specification guarantee that there needs to be a method named apply which matches the signature of the lambda, that is why we call it an implementation detail. Although this is indeed an interesting quirk, I would not rely on such code in any sort of production environment as it is subject to change.

Answer 2

I think I'm not fully getting exactly what the problem is. Your code seems to work as well.

But the line

I thought that lambda function in scala is also an object that extends the Function1

is correct if Scala version is < 2.12

Function literal in scala is just a syntactic sugar of FunctionN which are Function1 to Function22. So the below code is exactly the same.

val f: Int=> Int = new Function1[Int, Int] {
    def apply(x:Int): Int = x * x
}

val f2: Int => Int = (x: Int) => x * x

If you want to check both of them works as FunctionN,your code printValues should've be like this

def printValues(f: (Int) => Int, from: Int, to: Int): Unit = {
     println((from to to).map(f).mkString(" "))
}
//or
def printValues(f: Function1[Int,Int], from: Int, to: Int): Unit = {
     println((from to to).map(f).mkString(" "))
}

Once again, because of the syntactic sugar, they are the same meaning. Your definition of a parameter type is AnyRef{def apply(x: Int): Int} So

object Foo {
  def apply(x: Int): Int = x * x
}

can be used for printValues.

If you working on Scala 2.12, It's bit different story.

If your function literal meets some conditions, It'll be converted into SAM type automatically. For more detail, check this sam-conversion

EDITED You must be using 2.12 because the code work fine if it's on a 2.11 or eariler. like I mentioned in the last, It's all because of the sam-conversion. fix printValues's parameter into (Int) => Int or Function1[Int,Int] to define explicitly to use FunctionN