Spring Batch using Spring Boot - Never retry on write error

Question

I created a batch using Spring Boot. Here is the main configuration class of the batch :

@Configuration
@EntityScan({"my.domain"})
@EnableJpaRepositories({"my.domain"})
@EnableBatchProcessing
public class BatchConfiguration {

    /** Motif d'identification des fichiers d'entrée */
    @Value("${batch.input-file-pattern}")
    private String inputFilePattern;

    @Bean
    public BatchConfigurer configurer( EntityManagerFactory entityManagerFactory ){ 
        return new MapForcedBatchConfigurer( entityManagerFactory ); 
    }

    @Bean
    public Job myJob( JobBuilderFactory jobs, Step step1 ){
        return jobs.get("myJob")
                .incrementer( new RunIdIncrementer() )
                .flow( step1 )
                .end()
                .build();
    }

    @Bean
    public Step step1( StepBuilderFactory stepBuilderFactory, 
            StepExecutionListener stepExecutionListener,
            ItemReader<Input> myReader,
            ItemProcessor<Input, Dto> myProcessor, 
            ItemWriter<Dto> myWriter ){
        return stepBuilderFactory.get("myStep")
                .listener( stepExecutionListener )
                .<Input, Dto> chunk(1)
                .reader( myReader )
                .processor( myProcessor )
                .writer( myWriter )
                .faultTolerant().skipPolicy( new MySkipPolicy() ).retryLimit( 0 )
                .build();
    }

    @Bean
    public StepExecutionListener stepListener() {
        return new MyStepExecutionListener();
    }

    @Bean
    public ItemReader<Input> myReader() throws IOException {
        return new MyItemReader( inputFilePattern );
    }

    @Bean
    public ItemProcessor<Input, Dto> myProcessor(){
        return new MyItemProcessor();
    }

    @Bean
    public ItemWriter<Dto> myWriter(){
        return new MyItemWriter();
    }

}

When an error occurs processing an item, a logger write a message then the batch processes the next element. This is exactly what I wanted.

But when an error occurs writing an item, the batch always retry the operation once ! So that I have 2 error logs for each writing error.

How can I configure the batch to NEVER retry on error, no matter in which part of the step the errors occurs ?

This article explains the following :

[When we have a skip during writing], the framework has to find out which item caused the failure. For each item in the cached list of read items it starts an own transaction. The item is processed by the ItemProcessor and then written by the ItemWriter. If there is no error, the mini-chunk with one item is committed, and the iteration goes on with the next item. We expect at least one skippable exception, and when that happens, the transaction is rolled back and the item is marked as skipped item. As soon as our iteration is complete, we continue with normal chunk processing.

It could be an explanation of the re-execution of the writing operation. I hope I can bypass this behaviour in some way...

Answer 1

I finally came up with a pretty simple workaround. I moved the logs and operations in case of error in a method tagged @OnSkipInWrite. So that this code is only executed once, when Spring Batch finally skips the element causing the error.

public class MyItemWriter implements ItemWriter<Dto> {

    @Override
    public void write(List<? extends Dto> items) throws Exception {
        // The writing treatment which may throw a skippable exception...
    }

    @OnSkipInWrite
    public void onSkipInWrite( Dto skippedItem, Exception exception ){
        // Logs, counters, etc.
    }

}

It's not perfect beacuse the treatment throwing the exception is still executed twice. But I don't have twice the logs anymore and my custom error counts are right.

I'm still interested in a way to suppress this "retry once by mini-chunks" behaviour, even if I doubt it's possible.

Answer 2

I don't think there's much you can do without subclassing FaultTolerantChunkProcessor. Calling readerIsTransactionalQueue() on the step builder would force the behavior you want, but it would ignore all exceptions alike (would not invoke your skip policy).

The reason is that Spring Batch tries to determine the item that caused the error and even though your chunk only has one item, the algorithm doesn't take it into consideration (in fact, why are you using chunks of size 1?...)

What you can do is to catch the exception yourself in your writer. You could even write a wrapping ItemWriter for this sole purpose.