what's the right way to write scala.Function1 as a lambda?

Question

I am trying to transform() a Dataset in Java as follows:

Function1<Dataset<Long>,Dataset<Long>> withDoubled = (Dataset<Long> numbers) -> numbers.withColumn("doubled",numbers.col("id").multiply(2));
spark.range(10).transform(withDoubled).show();  

however, Function1<> gets flagged as an error saying there's multiple abstract functions to override. How do I write this as a lambda?

Answer 1

The use of lambda's with Scala's Function1 is not straigt forward, as lambda's work with Interfaces having only one abstract un-implemented function which is not true in case of Scala's Function1 trait.

We can use a work around for that,

First lets define builders which makes out work around reusable,

Builder for Function1,

package lambdascala;

import scala.Function1;

public class Function1WithLambdaBuilder<P1, R> {

  public static interface Function1LambdaApply<P1, R> {
    R apply(P1 p1);
  }

  private Function1LambdaApply<P1, R> lambda;

  private Function1<P1, R> function;

  public Function1WithLambdaBuilder(Function1LambdaApply<P1, R> lambda) {
    this.lambda = lambda;
    this.function = new Function1<P1, R> () {
      @Override
      public R apply(P1 p1) {
        return Function1WithLambdaBuilder.this.lambda.apply(p1);
      }
    };
  }

  public Function1<P1, R> getFunction() {
    return this.function;
  }

}

Another builder for Function2

package lambdascala;

import scala.Function2;

public class Function2WithLambdaBuilder<P1, P2, R> {

  public static interface Function2LambdaApply<P1, P2, R> {
    R apply(P1 p1, P2 p2);
  }

  private Function2LambdaApply<P1, P2, R> lambda;

  private Function2<P1, P2, R> function;

  public Function2WithLambdaBuilder(Function2LambdaApply<P1, P2, R> lambda) {
    this.lambda = lambda;
    this.function = new Function2<P1,P2, R> () {
      @Override
      public R apply(P1 p1, P2 p2) {
        return Function2WithLambdaBuilder.this.lambda.apply(p1, p2);
      }
    };
  }

  public Function2<P1, P2, R> getFunction() {
    return this.function;
  }

}

You can add builders for more FunctionN's following the same pattern.

Now we can use these builders to build Function1 and Function2

import lambdascala.Function1WithLambdaBuilder;
import lambdascala.Function2WithLambdaBuilder;
import scala.Function1;
import scala.Function2;
import java.util.List;

public class LambdaTry {

  public static void main() {

    Function1<List<Long>, List<Long>> changeNothing =
      new Function1WithLambdaBuilder<List<Long>, List<Long>>(
        // your lambda
        (List<Long> list) -> list
      ).getFunction();

    Function1<Integer, Integer> add2 =
      new Function1WithLambdaBuilder<Integer, Integer>(
        // your lambda
        (Integer i) -> i + 2
      ).getFunction();

    Function2<Integer, Integer, Integer> add =
      new Function2WithLambdaBuilder<Integer, Integer, Integer>(
        // your lambda
        (Integer i, Integer j) -> i + j
      ).getFunction();

    System.out.println(add2.apply(12));

    System.out.println(add.apply(12, 24));

  }

}