Spark: sum over list containing None and Some()?

Question

I already understand that I can sum over a list easily using List.sum:

var mylist = List(1,2,3,4,5)

mylist.sum
// res387: Int = 15

However, I have a list that contains elements like None and Some(1). These values were produced after running a left outer join.

Now, when I try to run List.sum, I get an error:

var mylist= List(Some(0), None, Some(0), Some(0), Some(1))

mylist.sum
<console>:27: error: could not find implicit value for parameter num: Numeric[Option[Int]]
       mylist.sum
              ^

How can I fix this problem? Can I somehow convert the None and Some values to integers, perhaps right after the left outer join?

Answer 1

You can use List.collect method with pattern matching:

mylist.collect{ case Some(x) => x }.sum
// res9: Int = 1

This ignores the None element.

---

Another option is to use getOrElse on the Option to extract the values, here you can choose what value you want to replace None with:

mylist.map(_.getOrElse(0)).sum
// res10: Int = 1

Answer 2

I find the easiest way to deal with a collection of Option[A] is to flatten it:

val myList = List(Some(0), None, Some(0), Some(0), Some(1))
myList.flatten.sum

The call to flatten will remove all None values and turn the remaining Some[Int] into plain old Int--ultimately leaving you with a collection of Int.

And by the way, embrace that immutability is a first-class citizen in Scala and prefer val to var.