Spring batch how to skip entire file on condition

Question

I have Spring batch job that reads some files and persists it to DB. If I need to be able to code some custom condition and skip file processing if it does not meet the condition. I've tried to extend ItemReader and throw exception but it caused whole job to fail while I need job to continue to iterate over files.

Thanks

Answer 1

Have a look to org.springframework.batch.core.step.skip.SkipPolicy interface. I give you an example extracted from Pro Spring Batch written by T. Minella

import java.io.FileNotFoundException;
import org.springframework.batch.core.step.skip.SkipLimitExceededException;
import org.springframework.batch.core.step.skip.SkipPolicy;
import org.springframework.batch.item.ParseException;

public class FileVerificationSkipper implements SkipPolicy {

public boolean shouldSkip(Throwable exception, int skipCount) throws SkipLimitExceededException {

if(exception instanceof FileNotFoundException) {
       return false;
} else if(exception instanceof ParseException && skipCount <= 10) {
       return true;
} else {
       return false;
    }
  }
}

Inside your xml file :

<step id="copyFileStep">
  <tasklet>
    <chunk reader="customerItemReader" writer="outputWriter"
commit-interval="10" skip-limit="10">
      <skippable-exception-classes>
        <include class="java.lang.Exception"/>
        <exclude class="org.springframework.batch.item.ParseException"/>
      </skippable-exception-classes>
    </chunk>
  </tasklet>
</step>

Or maybe another way could be to add at the beginning of your job a step wich sort your input files into two separate folder. Inside one folder you will have all your wrong files and inside the other folder only good ones remain.

Answer 2

So I ended up with extending MultiResourceItemReader and overriding read() method. Before delegate file to actual ItemReader, it validates the condition and pass file to reader only in case condition passed, otherwise processing to next file