Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Batch - how many records are read & processed

Is it possible to tell how many records are read and/or processed once the job is executed completely? I've a job that reads data from the database and in the processor, I filter few records based on certain criteria and send them to the writer. I would like to know how many total records are read from the DB and how many are sent to the writer step.

Here is my batch config file.

<bean id="dbItemReader" class="....JdbcCursorItemReader">
    <property name="datasource" ref="datasource"/>
    <property name="sql" ref="select * from"/>
    <property name="rowMapper">
      <bean class="com.my.MyRowMapper"/>
     </property> 
 </bean>

<bean id="itemProcessor" class="com.my.MyItemProcessor"/> 
<bean id="itemWriter" class="com.my.MyItemWriter"/>

<batch:job id="myJob">
    <batch:step id="step1">
       <batch:tasklet transaction-manager="jobTransactionManager">
             <batch:chunk reader="dbItemReader" processor="itemProcessor" writer="itemWriter" commit-interval="100"/>
       </batch:tasklet> 

like image 282
RKodakandla Avatar asked May 21 '15 03:05

RKodakandla


People also ask

Is Spring Batch single threaded?

Multithreaded steps. By default, Spring Batch uses the same thread to execute a batch job from start to finish, meaning that everything runs sequentially. Spring Batch also allows multithreading at the step level. This makes it possible to process chunks using several threads.

What are listeners in Spring Batch?

Spring Batch listeners are a way of intercepting the execution of a Job or a Step to perform some meaningful operations or logging the progress.

What is filter count in Spring Batch?

If an item was skipped for example, it would have been read, but not filtered or written. For the record, the filter count is the count of times the ItemProcessor returned null which is different than an item being skipped due to a skippable exception being thrown.

How does Spring Batch handle transactions?

Spring Batch handles transactions at the step level. This means that Spring Batch will never use only one transaction for a whole job (unless the job has a single step). Remember that you're likely to implement a Spring Batch job in one of two ways: using a tasklet or using a chunk-oriented step.


1 Answers

Spring Batch stores the number of items read, processed, skipped, written, etc in the job repository. Assuming you're using a database job repository, you can view them there in the BATCH_STEP_EXECUTION table.

You can read more about the information stored in the job repository in the documentation here: http://docs.spring.io/spring-batch/reference/html/metaDataSchema.html

like image 160
Michael Minella Avatar answered Oct 09 '22 13:10

Michael Minella