Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to groupBy an iterator without converting it to list in scala?

Suppose I want to groupBy on a iterator, compiler asks to "value groupBy is not a member of Iterator[Int]". One way would be to convert iterator to list which I want to avoid. I want to do the groupBy such that the input is Iterator[A] and output is Map[B, Iterator[A]]. Such that the part of the iterator is loaded only when that part of element is accessed and not loading the whole list into memory. I also know the possible set of keys, so I can say whether a particular key exists.

def groupBy(iter: Iterator[A], f: fun(A)->B): Map[B, Iterator[A]] = {
    .........
}
like image 530
Arun Avatar asked Oct 20 '22 15:10

Arun


2 Answers

One possibility is, you can convert Iterator to view and then groupBy as,

iter.toTraversable.view.groupBy(_.whatever)
like image 140
S.Karthik Avatar answered Oct 22 '22 21:10

S.Karthik


I don't think this is doable without storing results in memory (and in this case switching to a list would be much easier). Iterator implies that you can make only one pass over the whole collection.

For instance let's say you have a sequence 1 2 3 4 5 6 and you want to groupBy odd an even numbers:

groupBy(it, v => v % 2 == 0)

Then you could query the result with either true and false to get an iterator. The problem should you loop one of those two iterators till the end you couldn't do the same thing for the other one (as you cannot reset an iterator in Scala).

This would be doable should the elements were sorted according to the same rule you're using in groupBy.

like image 39
Mateusz Dymczyk Avatar answered Oct 22 '22 22:10

Mateusz Dymczyk