Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scala Spark contains vs. does not contain

I can filter - as per below - tuples in an RDD using "contains". But what about filtering an RDD using "does not contain" ?

val rdd2 = rdd1.filter(x => x._1 contains ".")

I cannot find the syntax for this. Assuming it is possible and that I'm not using DataFrames. I cannot see from how to do it with regex and/or filter examples.

like image 526
thebluephantom Avatar asked Nov 05 '16 14:11

thebluephantom


1 Answers

It's just the negation of the contains filter predicate :

val rdd2 = rdd1.filter(x => !(x._1 contains "."))
like image 83
eliasah Avatar answered Oct 23 '22 21:10

eliasah