How do I kickoff a batch job when input file arrives?

Question

We have Spring4 and Spring Batch 3 and our app consumes CSV files as input file. Currently we kick off the jobs manually from the command line, using CommandLineJobRunner with parms, including the name of the file to process.

I want to kick off a job to process asynchronously just as soon as the input file arrives in a monitored directory. How can we do that?

Answer 1

You may use java.nio.file.WatchService to monitor directory for a file. Once file appears you may start (or kick off a job to process asynchronously) actual processing.

You may also use FileReadingMessageSource.WatchServiceDirectoryScanner from Spring Integration (https://docs.spring.io/spring-integration/reference/html/files.html#watch-service-directory-scanner)

Comparing release notes Spring Batch https://github.com/spring-projects/spring-batch/releases to Spring Integration https://github.com/spring-projects/spring-integration/releases it looks that Spring Integration is released more often. It also has more features and Integration points.

In this case it looks like a overkill to bring Spring Integration if you just need to watch a directory for a file.

Answer 2

I would recommend using the powerful combination of Spring Batch with Spring Integration. For example, you can use a FileInboundChannelAdapter from Spring Integration to monitor a directory and start a Spring Batch Job as soon as the input file arrives.

There is a code example for this typical use case in the reference documentation of Spring Batch here: https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-integration.html#launching-batch-jobs-through-messages

I hope this helps.