Repeat steps of a Spring Batch flow for each item of a list

Question

How can I implement a Spring Batch job which has to read a list and then repeat one or more steps for each item in the list?

I am currently reading the list in the one step and then I put it in the job context. But the job context is persisted in the DB and if it gets too big, a CLOB has to be used and I do not have access to one.

So I am looking for a solution that doesn't involve storing the whole list on the job context.

Of course, I could simply put the list in a local variable. But I am curious whether there is a more Spring Batch-like option.

Answer 1

Aside from the comments above about structuring the job in the first place (which I tend to agree with), if you use the latest 3.0.0.M3, you can create a JobScope'ed container that could hold the collection as you loop through the various steps. From there you can read/process/write to that container instead of an external source.

Answer 2

As discussed on Spring Batch-Repeat step for each item in a data list question, Partitioning seems to be the solution.

Configuration for PartitionHandler:

        <batch:step id="anyNameJobMaster">
            <batch:partition step="nameOfActualStepToBeIterated" partitioner="partitioner">
                <batch:handler grid-size="10" task-executor="taskExecutor" />
            </batch:partition>
        </batch:step>

       <bean id="partitioner" class="org.example.PartitionerThatImplementsPartitioner" lazy-init="true" scope="step"/>

PartitionHandler Sample Code:

public class PartitionerThatImplementsPartitioner implements Partitioner{
    @Override
    public Map<String, ExecutionContext> partition(int gridSize) {
         Map<String, ExecutionContext> map = new HashMap<String, ExecutionContext>(gridSize);  
         //Iteration goes here
         //For Each Iteration, create ExecutionContext and add it to map.
         map.put("somePartionKeyString", new ExecutionContext().put("ContextIdentifiere.g.FileName", "IteratedVale.g.FilePath"))

    }
}

Job Configuration:

     <batch:step id="nameOfActualStepToBeIterated">
        <batch:tasklet transaction-manager="someTxManager">
            <batch:chunk processor="someProcessor"
                writer="itemWriter" commit-interval="${commitInterval}">
                <batch:reader>
                    <bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
                        <property name="resource" value="#{stepExecutionContext['ContextIdentifiere.g.FileName']}" />                       
                    </bean>
                </batch:reader>
            </batch:chunk>
        </batch:tasklet>