Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

spring batch ItemReader FlatFileItemReader set cursor to start reading from a particular line or set linestoskip dynamically

In my springbatch+quartz setup, I am reading a CSV File using FlatFileItemReader. I want to set the cursor for the reader to start the next jobinstance with the given parameters for reader. Is it possible?

<bean id="cvsFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <!-- Read a csv file -->
    <property name="resource" value="classpath:cvs/input/report.csv" />

    <property name="lineMapper">
        <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean
                    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="names" value="id,impressions" />
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean
                    class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                    <property name="prototypeBeanName" value="report" />
                </bean>
            </property>
        </bean>
    </property>

</bean>

The idea is to continue reading the file where last failure occured in the next execution. I am putting an integer 'writecursor' for each line written in my customWriter.

public void write(List<? extends Report> items) throws Exception {

System.out.println("writer..." + items.size() + " > ");     
for(Report item : items){
    System.out.println("writing item id: " + item.getId());     
    System.out.println(item);

}
//getting stepExecution by implementing StepExecutionListener
this.stepExecution.getExecutionContext().putInt("writecursor", ++writecursor);

}

Now, in the customItemReadListener, I want to get the update writecursor value and then skip the lines from the top to start reading from writecursor

public class CustomItemReaderListener implements ItemReadListener<Report>, StepExecutionListener {

    ApplicationContext context = ApplicationContextUtils.getApplicationContext();
    private StepExecution stepExecution;
    @Override
    public void beforeRead() {
        //Skip lines somehow
    }

Another thing I saw as a possible solution is to set linestoskip dynamically in itemreader. There is a thread here http://thisisurl.com/dynamic-value-for-linestoskip-in-itemreader but not answered yet. And here, http://forum.spring.io/forum/spring-projects/batch/100950-accessing-job-execution-context-from-itemwriter

like image 571
UserBSS1 Avatar asked Mar 21 '23 17:03

UserBSS1


1 Answers

Use FlatFileItemReader.linesToSkip property setted injecting job Parameter value.

<bean id="myReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    <property name="linesToSkip" value="file:#{jobParameters['cursor']}" />
</bean>
like image 89
Luca Basso Ricci Avatar answered Apr 26 '23 20:04

Luca Basso Ricci