I have a Stream of MyObject that I want batch persist into DB (not one by one but let's say 1000 at once). So I want to do a transformation, something like
Stream<MyObject> ---> Stream<List<MyObject>>
where each List has some fixed size batchSize. Is there a way to do that with standard Java 8 Stream API?
Edit: the original solution below does not work, since java stream does not allow calling skip or limit more than once on a same stream. I ended up simple processing like
final AtomicInteger counter = new AtomicInteger();
List<T> entityBatch = new ArrayList<>();
entityStream.forEach(entity -> {
if (counter.intValue() = batchSize) {
processBatch(entityBatch);
entityBatch.clear();
counter.set(0);
}
entityBatch.add(entity);
counter.incrementAndGet();
});
if (!entityBatch.isEmpty()) {
processBatch(entityBatch);
}
Original solution: It looks like I found the way to do that:
<T> Stream<List<T>> batchStream(Stream<T> stream, int batchSize) {
return Stream.iterate(stream, s -> s.skip(batchSize)).map(s -> s.limit(batchSize).collect(toList()));
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With