I have an ordered Spark DataFrame
and I would like to change a few rows while iterating it using the following code but it seems there is not any way to update Row object.
orderedDataFrame.foreach(new Function1<Row,BoxedUnit>(){
@Override
public BoxedUnit apply(Row v1) {
// How do I change Row here?
// I want to change column no 2 using v1.get(2)
// also what is BoxedUnit, and how do I use it
return null;
}
});
Also above code is giving compilation error saying:
myclassname is not abstract and it does not override abstract method apply$mcVj$sp(long) in scala Function 1
I am new to Spark. I am using 1.4.0 release.
Try This:
final DataFrame withoutCurrency = sqlContext.createDataFrame(somedf.javaRDD().map(row -> {
return RowFactory.create(row.get(0), row.get(1), someMethod(row.get(2)));
}), somedf.schema());
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With