How to Schedule a Spring Boot Batch Application

Question

I have a Spring Boot Batch application:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class LoadApplication {

    public static void main(String[] args) {
        SpringApplication.run(LoadApplication.class, args);
    }
}

In a @Configuration class also annotated with @EnableBatchProcessing, I have the following batch job bean defined:

@Bean
public Job loadJob(JobBuilderFactory jobs, Step stepLoadFile, Step stepArchiveFile) {
    return jobs.get("loadJob")
            .incrementer(new RunIdIncrementer())
            .start(stepLoadFile)
            .next(stepArchiveFile)
            .build();
}

The batch job's stepLoadFile reads a flat file (see below) and writes the file data to a database. stepArchiveFile then moves the file to a different directory.

Normally, the job needs to run once a day (Tuesday - Saturday) at a specified time. However, if the flat file is not found, then the job fails and needs to rerun every 30 minutes until it succeeds or a max number (e.g. 5) of attempts is hit. Once a rerun is successful, the job should not run again until the next normal run time. Also, the system should ideally prevent concurrent runs of the same job. How can all of this be done?

Note: The rerun does not need to pick up where the previous job run failed. This is because the chunk size is set larger than the number of items in the file.

I tried this in my @Configuration class (Note: I also added @EnableRetry to the configuration and main class):

@Bean
public ItemReader<Test> reader(LineMapper<Test> lineMapper, ApplicationProperties properties) {
    FlatFileItemReader<Test> flatFileItemReader = new FlatFileItemReader<Test>() {
        @Override
        @Retryable(value = {ItemStreamException.class}, maxAttempts=5)
        public void open(ExecutionContext executionContext) throws ItemStreamException {
            super.open(executionContext);
        }

        @Override
        @Retryable(maxAttempts=5)
        public Holding read() throws UnexpectedInputException, ParseException, Exception {
            return super.read();
        }

    };
    flatFileItemReader.setResource(new FileSystemResource(properties.getLoadFile()));
    flatFileItemReader.setLineMapper(lineMapper);
    return flatFileItemReader;
}

ItemStreamException is thrown and the application exits without retrying.

Answer 1

You can schedule by adding the below component in LoadApplication class

@Component
@EnableScheduling
class ScheduledTasks {
@Autowired
JobLauncher jobLauncher;

@Autowired
Job job;

/**The pattern is a list of six single space-separated fields: 
 * representing second, minute, hour, day, month, weekday. 
 * Month and weekday names can be given as the first three letters of the English names.
 * Example patterns:

    "0 0 * * * *" = the top of every hour of every day.*
    "*\/10 * * * * *" = every ten seconds. Remove 2nd character, it is escape
    "0 0 8-10 * * *" = 8, 9 and 10 o'clock of every day.
    "0 0/30 8-10 * * *" = 8:00, 8:30, 9:00, 9:30 and 10 o'clock every day.
    "0 0 9-17 * * MON-FRI" = on the hour nine-to-five weekdays
    "0 0 0 25 12 ?" = every Christmas Day at midnight

 */
@Scheduled(cron = "0 0/30 * * * TUE-SAT")
public void runJob() throws Exception {
    jobLauncher.run(job, new JobParameters());
}
}

For retrying failed steps you may have to configure it in job step itself
<chunk reader="itemReader" writer="itemWriter" commit-interval="2" retry-limit="5"> <retryable-exception-classes> <include class="java.io.FileNotFoundException"/> </retryable-exception-classes> </chunk>

Also if you store spring-batch meta-data tables on disc rather than in memory storage, spring batch wont run the same job again.