Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Batch Reader in Multi threader Job

Tags:

spring-batch

I'm Using Spring Batch to run my JOBS when the reader is org.springframework.batch.item.database.JpaPagingItemReader

And my JOB configured with the parameter: throttle-limit="6"

Is there any threads data conflict when reading data from the reader?

why i'm reviving the following warning:

[org.springframework.batch.core.step.item.ChunkMonitor:109] - No ItemReader set (must be   concurrent step), so ignoring offset data.
[org.springframework.batch.core.step.item.ChunkMonitor:141] - ItemStream was opened in a different thread.  Restart data could be compromised.
[org.springframework.batch.core.step.item.ChunkMonitor:141] - ItemStream was opened in a different thread.  Restart data could be compromised.
[org.springframework.batch.core.step.item.ChunkMonitor:141] - ItemStream was opened in a different thread.  Restart data could be compromised.
like image 898
Yosefarr Avatar asked May 21 '14 10:05

Yosefarr


People also ask

How does multithreading work in Spring Batch?

Multithreaded steps. By default, Spring Batch uses the same thread to execute a batch job from start to finish, meaning that everything runs sequentially. Spring Batch also allows multithreading at the step level. This makes it possible to process chunks using several threads.

Can a Spring Batch have multiple jobs?

Multiple jobs can be run simultaneously. There are two main types of Spring Batch Parallel Processing: Single Process, Multi-threaded, or Multi-process.

How does Spring Batch reader work?

An Item Reader reads data into the spring batch application from a particular source, whereas an Item Writer writes data from Spring Batch application to a particular destination. An Item processor is a class which contains the processing code which processes the data read in to the spring batch.


1 Answers

While the JpaPagingItemReader is thread safe in that using it to read from multiple threads is ok, that reader will not support restart when used via multiple threads. The warnings are really there to indicate that you need to set the save state to false (JpaPaginingItemReader#setSaveState(boolean)). The documentation for this class discusses the need to set save state to false here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/database/JpaPagingItemReader.html

The reason for this is that the restart state ends up being shared across all threads so they end up stepping on each other.

like image 102
Michael Minella Avatar answered Sep 20 '22 12:09

Michael Minella