Sortby in Javardd

Question

I'm usinig spark with java. And i want to sort my map. In fact, i have i javaRDD like this :

JavaPairRDD<String, Integer> rebondCountURL = session_rebond_2.mapToPair(new PairFunction<Tuple2<String, String>, String, String>() {
        @Override
        public Tuple2<String, String> call(Tuple2<String, String> stringStringTuple2) throws Exception {
            return new Tuple2<String, String>(stringStringTuple2._2, stringStringTuple2._1);
        }
    }).groupByKey().map(new PairFunction<Tuple2<String, Iterable<String>>, Tuple2<String, Integer>>() {
        @Override
        public Tuple2<String, Integer> call(Tuple2<String, Iterable<String>> stringIterableTuple2) throws Exception {
            Iterable<String> strings = stringIterableTuple2._2;
            List<String> b = new ArrayList<String>();
            for (String s : strings) {
                b.add(s);
            }
            return new Tuple2<String, Integer>(stringIterableTuple2._1, b.size());
        }
    });

And i want to sort this Java Rdd using Sortby (in order to sort using the Integer).

Can you help me please to do it ?

Thank you in advance.

Answer 1

You need to create a function which extracts the sorting key from each element. Example from our code

final JavaRDD<Something> stage2 = stage1.sortBy( new Function<Something, Long>() {
  private static final long serialVersionUID = 1L;

  @Override
  public Long call( Something value ) throws Exception {
    return value.getTime();
  }
}, true, 1 );