How to test the whole flow of a Spring Batch application?

Question

I have a Spring boot + batch application which reads a source CSV file, process it and write to target CSV file, I'm struggling with writing tests that will: use an input - "simpleFlowInput.csv" and compare the "simpleFlowActual.csv" output with an "simpleFlowExpected.csv" file, i would like to write many of these tests but struggle with the way to do it.

My application contain only one step and one job:

@Bean("csvFileToFileStep")
public Step csvFileToFileStep() {
    return stepBuilderFactory.get("csvFileToFileStep").<RowInput, RowOutput>chunk(10000).reader(csvRowsReader()).processor(csvRowsProcessor())
            .writer(compositeItemWriter()).build();
}

@Bean("csvFileToCsvJob")
Job csvFileToCsvJob(JobCompletionNotificationListener listener) {
    return jobBuilderFactory.get("csvFileToCsvJob").incrementer(new RunIdIncrementer()).listener(listener).flow(csvFileToFileStep()).end()
            .build();
}

My current test:

@RunWith(SpringJUnit4ClassRunner.class)
@Configuration
@EnableBatchProcessing
@SpringBootTest
public class Tester{

    @Autowired
    Job csvFileToCsvJob;
    @Autowired
    Step csvFileToFileStep;
    @Autowired
    CsvFileReadProcessAndWriteConfig csvFileReadProcessAndWriteConfig;
    private JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();


    @Test
    public void testSimpleFlow() throws Exception {

        ClassLoader classLoader = getClass().getClassLoader();
        File fileInput = new File(classLoader.getResource("simpleFlowInput.csv").getFile());
        File fileActual = new File(classLoader.getResource("simpleFlowActual.csv").getFile());
        File fileExpected = new File(classLoader.getResource("simpleFlowExpected.csv").getFile());

        FileManager.getInstance().setInputFileLocation(fileInput.toString());
        FileManager.getInstance().setOutputFileLocation(fileActual.toString());

        System.out.println(fileExpected.length());
        System.out.println(fileActual.length());

        Assert.assertTrue(fileExpected.length() == fileActual.length());
        AssertFile.assertFileEquals(fileExpected,fileActual);//compare
    }
}

Any advise on how to test it ?

( I found this question written at 2010 with a partial answer mentioning "JobLauncherTestUtils". What is the best way to test job flow in Spring-Batch? )

Answer 1

The End-To-End Testing of Batch Jobs section of the documentation explains in details how to test a Spring Batch job (including how to use the JobLauncherTestUtils).

Spring Batch provides a nice utility class called AssertFile in the spring-batch-test module which can be helpful in your case: You write the expected file and then assert the actual one (generated by your job) against it. The section Validating Output Files shows how to use this class.

Hope this helps.