Menu
I have a Stream processing a few millions of elements. The Map-Reduce algorithm behind it takes a few milliseconds, so task completion takes about twenty minutes.
Stream<MyData> myStream = readData(); MyResult result = myStream .map(row -> process(row)) .peek(stat -> System.out.println("Hi, I processed another item")) .reduce(MyStat::aggregate); I'd like a way to display overall progress, instead of printing a line per element (which results in thousands of lines per second, takes time and doesn't provide any useful information regarding overall progress). I would like to display something similar to:
5% (08s) 10% (14s) 15% (20s) ... What would be the best (and/or easiest) way to do this?
594
In Java8 Streams, performance is achieved by parallelism, laziness, and using short-circuit operations, but there is a downside as well, and we need to be very cautious while choosing Streams, as it may degrade the performance of your application. Let us look at these factors which are meant for Streams' performance.
Streams are lazy because intermediate operations are not evaluated unless terminal operation is invoked. Each intermediate operation creates a new stream, stores the provided operation/function and return the new stream.
Yes, streams are sometimes slower than loops, but they can also be equally fast; it depends on the circumstances. The point to take home is that sequential streams are no faster than loops.
Introduced in Java 8, the Stream API is used to process collections of objects. A stream is a sequence of objects that supports various methods which can be pipelined to produce the desired result.
First of all, Streams are not meant to achieve these kind of tasks (as opposed to a classic data structure). If you know already how many elements your stream will be processing you might go with the following option, which is, I repeat, not the goal of streams.
Stream<MyData> myStream = readData(); final AtomicInteger loader = new AtomicInteger(); int fivePercent = elementsCount / 20; MyResult result = myStream .map(row -> process(row)) .peek(stat -> { if (loader.incrementAndGet() % fivePercent == 0) { System.out.println(loader.get() + " elements on " + elementsCount + " treated"); System.out.println((5*(loader.get() / fivePercent)) + "%"); } }) .reduce(MyStat::aggregate); If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
