Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I break a collection into batches?

I have a simple task here: break a Set of n elements into m Sets based on a batch size - typically I'll want to limit my sub-Sets to 1,000 elements. I wrote something like this, where input is the master, large collection:

var strings = Set[String]() ++ input
var sets = List[Set[String]]()
while (!strings.isEmpty) {
  val (head, rest) = strings.splitAt(100)
  sets = sets :+ head
  securities = rest
}

which works fine, but I am thinking there HAS to be a more elegant/functional solution to such a simple and common problem in Scala. Someone please enlighten me.

like image 815
Scala Newb Avatar asked May 23 '13 14:05

Scala Newb


2 Answers

And it is exists: .grouped(batchSize). Example:

scala> List.range(1,10).toSet.grouped(3).toList
// res0: List[scala.collection.immutable.Set[Int]] = List(
//    Set(5, 1, 6), 
//    Set(9, 2, 7), 
//    Set(3, 8, 4))
like image 149
om-nom-nom Avatar answered Oct 18 '22 03:10

om-nom-nom


Just call Set(1,2,3).grouped(1).toList

scala> Set(1,2,3).grouped(1).toList
res1: List[scala.collection.immutable.Set[Int]] = List(Set(1), Set(2), Set(3))
like image 42
4lex1v Avatar answered Oct 18 '22 02:10

4lex1v