Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Batch Item Reader is executing only once

Trying to implement Spring batch,but facing a strange problem,Our ItemReader class is executing only once.

Here below is the detail.

  1. If we have 1000 rows in DB.
  2. Our Item reader fetch 1000 rows from DB,and pass list to ItemWriter
  3. ItemWriter successfully delete all items.
  4. Now ItemReader again tries to fetch the data from DB,but did not find,hence returns NULL,so execution stops.
  5. But we have configured batch to be executed with Quartz scheduler,which is every minute.
  6. Now if we insert let say 1000 rows in DB by dump import,the batch job should pick this data in next execution,but it is not even executing,although JobLauncher is executing.

Configuration :-

1.We have ItemReader,ItemWriter with commit interval equals to 1.

<batch:job id="csrfTokenBatchJob">
    <batch:step id="step1">
      <tasklet>
        <chunk reader="csrfTokenReader" writer="csrfTokenWriter" commit-interval="1"></chunk>
      </tasklet>
    </batch:step>
  </batch:job>

2.Job is scheduled to be triggered at every minute.

<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
    <property name="triggers">
      <bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
        <property name="jobDetail" ref="jobDetail" />
        <property name="cronExpression" value="0 0/1 * * * ?" />
      </bean>
    </property>
  </bean>

3.Job configuration

<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
    <property name="jobClass" value="com.tavant.oauth.batch.job.CSRFTokenJobLauncher" />
    <property name="jobDataAsMap">
        <map>
            <entry key="jobName" value="csrfTokenCleanUpBatchJob" />
            <entry key="jobLocator" value-ref="jobRegistry" />
            <entry key="jobLauncher" value-ref="jobLauncher" />
        </map>
    </property>
</bean>

First time it is executing successfully,but later it does not execute,but i can see in logs that JobLauncher is executing.

@Component("csrfTokenReader")
@Scope(value="step")
public class CSRFTokenReader implements ItemReader<List<CSRFToken>> {

    private static final Logger logger = LoggerFactory.getLogger(CSRFTokenReader.class);

    @Autowired
    private CleanService cleanService;

    @Override
    public List<CSRFToken> read() {
        List<CSRFToken> csrfTokenList = null;
        try{

            int keepUpto = Integer.valueOf(PropertiesContext.getInstance().getProperties().getProperty("token.keep", "1"));

            Calendar calTime = Calendar.getInstance();
            calTime.add(Calendar.HOUR, -keepUpto);
            Date toKeep = calTime.getTime();

            csrfTokenList = cleanService.getCSRFTokenByTime(toKeep);
        }
        catch(Throwable th){
            logger.error("Exception in running job At " + new Date() + th);
        }
        if(CollectionUtils.isEmpty(csrfTokenList)){
            return null;
        }
        return csrfTokenList;
    }
}

EDIT:--

public class CSRFTokenJobLauncher extends QuartzJobBean {
    static final String JOB_NAME = "jobName";
    private JobLocator jobLocator;
    private JobLauncher jobLauncher;
    public void setJobLocator(JobLocator jobLocator) {
        this.jobLocator = jobLocator;
    }
    public void setJobLauncher(JobLauncher jobLauncher) {
        this.jobLauncher = jobLauncher;
    }
    @Override
    protected void executeInternal(JobExecutionContext context) {
        Map<String, Object> jobDataMap = context.getMergedJobDataMap();
        String jobName = (String) jobDataMap.get(JOB_NAME);
        log.info("Quartz trigger firing with Spring Batch jobName="+jobName);
        JobParameters jobParameters = getJobParametersFromJobMap(jobDataMap);
        try {
            jobLauncher.run(jobLocator.getJob(jobName), jobParameters);
        }
        catch (JobExecutionException e) {
            log.error("Could not execute job.", e);
        }
    }
    private JobParameters getJobParametersFromJobMap(Map<String, Object> jobDataMap) {
        JobParametersBuilder builder = new JobParametersBuilder();
        for (Entry<String, Object> entry : jobDataMap.entrySet()) {
            String key = entry.getKey();
            Object value = entry.getValue();
            if (value instanceof String && !key.equals(JOB_NAME)) {
                builder.addString(key, (String) value);
            }
            else if (value instanceof Float || value instanceof Double) {
                builder.addDouble(key, ((Number) value).doubleValue());
            }
            else if (value instanceof Integer || value instanceof Long) {
                builder.addLong(key, ((Number)value).longValue());
            }
            else if (value instanceof Date) {
                builder.addDate(key, (Date) value);
            }
        }
        return builder.toJobParameters();
    }
}
like image 585
dReAmEr Avatar asked May 11 '15 10:05

dReAmEr


2 Answers

After hours of time wasting,the problem seems to be solved now,i have configured allow-start-if-complete="true" in tasklet.Now Batch Item Reader is executing as per schedule.

<batch:job id="csrfTokenBatchJob">
    <batch:step id="step1">
      <batch:tasklet allow-start-if-complete="true">
        <batch:chunk reader="csrfTokenReader" writer="csrfTokenWriter" commit-interval="1"></batch:chunk>
      </batch:tasklet>
    </batch:step>
  </batch:job>
like image 186
dReAmEr Avatar answered Nov 19 '22 17:11

dReAmEr


Spring batch records every job execution in database. Which is why spring batch need to differentiate every job run. It checks whether the job is already executed on the same day and it would not start again unless any job parameter varies from previous run or allow start if complete setting is enabled.

OPTION1:- As mentioned above answer we can use allow-start-if-complete="true"

OPTION2:- Always pass a job parameter which is a current date time stamp. This way job parameter value is always unique.

    JobExecution jobExecution = jobLauncher.run(reportJob, new JobParametersBuilder()
                    .addDate("now", new Date()).build());

OPTION3:- Use an incrementor for example RunIdIncrementer so we do not need to make sure to pass unique job parameter every time.

    @Bean
    public Job job1(JobBuilderFactory jobs, Step s1) {
        return jobs.get("job1")
                .incrementer(new RunIdIncrementer())
                .flow(s1)
                .end()
                .build();
    }
like image 38
Sushil Behera Avatar answered Nov 19 '22 15:11

Sushil Behera