Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Chunk oriented processing multithreaded?

If I define like this for spring batch:

<chunk reader="chunkReader" 
    writer="chunkWriter" 
    processor="chunkProcessor" 
    commit-interval="#{jobParameters['commitSize']}" />

In this spring batch's chunk-oriented processing, are the chunks processed in parallel? And are the individual items in the chunks processed in parallel?

I am asking mainly to see if I need to worry about multithreading and race conditions.

like image 605
Neeraj Gupta Avatar asked Sep 13 '26 14:09

Neeraj Gupta


1 Answers

Unless you instruct Spring, it will be all sequential.

If you decide to use multithreading, a batch job can use Spring’s TaskExecutor abstraction to execute each chunk in its own thread. A step in a job can be configured to perform within a threadpool, processing each chunk independently. As chunks are processed, Spring Batch keeps track of what is done accordingly. If an error occurs in any one of the threads, the job’s processing is rolled back or terminated per the regular Spring Batch functionality.

See: https://docs.spring.io/spring-batch/trunk/reference/html/scalability.html

and

How to set up multi-threading in Spring Batch?

like image 131
gargkshitiz Avatar answered Sep 15 '26 10:09

gargkshitiz



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!