Menu
I am trying to get fancy with my batch jobs and utilize a progress bar for my batch jobs, like the one here: https://github.com/ctongfei/progressbar
To accurately populate a progress bar like the one linked above, however, I would need to know the total number of chunks/elements to be processed in the job.
I am looking to add this in my ChunkListener. Within here, I am able to grab my count of items read -> processed -> written thru the ChunkContext:
context.getStepContext().getStepExecution().getWriteCount()
What I cannot figure out, however, is how to get the total number of chunks or total number of items in the step, while the step is in progress. Does Spring Batch have this capability? If so, how do I get this value?
515
Some books like "Spring Batch in Action" recommend to keep the chunk size commonly between 20 to 200. Some ideas from the same book are as follows: Too small a chunk size creates too many transactions, which is costly and makes the job run slowly.
In the above code, the chunk size is set to 5, the default batch chunk size is 1. So, it reads, processes, and writes 5 of the data set each time. The reader can be defined by using the ItemReader interface which comes from the Spring Batch framework.
Spring Batch uses a 'Chunk-oriented' processing style within its most common implementation. Chunk oriented processing refers to reading the data one at a time and creating 'chunks' that are written out within a transaction boundary.
Chunk oriented processing refers to reading the data one at a time, and creating 'chunks' that will be written out, within a transaction boundary. One item is read in from an ItemReader , handed to an ItemProcessor , and aggregated.
Spring Batch does not provide this capability as it heavily depends on the input data. You need to calculate the total number of records upfront and use that information to calculate the progress during the step execution.
Calculating the total number of records can be done using a job/step listener or a tasklet for example which puts the information in the execution context. Then, an ItemReadListener can be used to calculate the progress based on the current item count and the total item count.
198
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
