Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring Batch retry-policy, and skip-policy issue

I have following step in batch job.

    <batch:step id="parse-step">
        <batch:tasklet>
            <batch:chunk reader="xmlCommonReader"
                         processor="xmlCommonProcessor"
                         writer="xmlCommonWriter"
                         commit-interval="1">
                <batch:skip-policy>
                    <bean class="org.springframework.batch.core.step.skip.AlwaysSkipItemSkipPolicy" scope="step"/>
                </batch:skip-policy>
                <batch:retry-policy>
                    <bean class="org.springframework.retry.policy.NeverRetryPolicy" scope="step"/>
                </batch:retry-policy>
            </batch:chunk>
        </batch:tasklet>
        <batch:next on="FAILED" to="file-failed-step"/>
        <batch:next on="COMPLETED" to="file-success-step"/>
        <batch:listeners>
            <batch:listener ref="parseStepExecutionListener"/>
            <batch:listener ref="parseStepSkipListener"/>
        </batch:listeners>
    </batch:step>

When some exception throws, i catch him in parseStepSkipListener and log in database.

I expect the following behavior:

  1. Job Started
  2. Executing previous steps
  3. Start execution of parse-step
  4. Read item
  5. Process item
  6. Write
    1. Ooooops, exception.
    2. Catch exception, log in database, go to next chunk(Read, Process, Write).
  7. Continue execute other steps.
  8. Finish job.

But actually i get following behavior:

  1. Job Started
  2. Executing previous steps
  3. Start execution of parse-step
  4. Read item
  5. Process item
  6. Write
    1. Ooooops, exception.
    2. Process item
    3. Write item
      1. Ooooops, exception.
      2. Catch exception, log in database, go to next chunk(Read, Process, Write).
  7. Continue execute other steps.
  8. Finish job.

So, one chunk of data try to process and write two times.

like image 208
Ruslan Avatar asked Mar 10 '14 10:03

Ruslan


People also ask

How do I specify a retry policy in Spring Batch?

As with the skip functionality, there are two ways of specifying retry behaviour in Spring Batch. The convenient standard way would be specifying a retry-limit on the chunk and nesting retryable-exception-classes inside the chunk: As with skipping, you may specify your own RetryPolicy and plug it into the chunk:

What does Spring Batch skip and retry do?

Spring Batch is a great framework offering functionality for complex processings like skipping or retrying failed items, but you still need to understand what Spring Batch does to avoid problems. In this article we saw potential stumbling blocks when using skip and retry functionality.

What happens when a Spring Batch job fails?

Overview By default, a Spring batch job fails for any errors raised during its execution. However, at times, we may want to improve our application's resiliency to deal with intermittent failures. In this quick tutorial, we'll explore how to configure retry logic in the Spring Batch framework.

Is there a rollback in Spring Batch?

There’s no rollback. As with the skip functionality, there are two ways of specifying retry behaviour in Spring Batch. The convenient standard way would be specifying a retry-limit on the chunk and nesting retryable-exception-classes inside the chunk: As with skipping, you may specify your own RetryPolicy and plug it into the chunk:


1 Answers

In few words:

This happens because when an error occured in write step SB doesn't know which object caused the exception so a rollback is performed and every single item of last uncommited chunk is processed/writed again as a mini-chunk to detect which object was the cause of main write error. You can read more (with diagrams) here

like image 139
Luca Basso Ricci Avatar answered Sep 21 '22 10:09

Luca Basso Ricci