Spring Batch serialization problem when using Jackson

Question

While upgrading an application from Spring Boot 1.3.x to 2.1.x I encountered a problem with Spring Batch. Apparently, Spring Batch needs to serialize its execution context and used XStream so far, but switched to Jackson sometime in the version history.

Now, I get an exception when running a job:

2019-10-08 15:50:58:630 ERROR [main] [:] o.s.b.c.j.AbstractJob [] Encountered fatal error executing job
    org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:142) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) [spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    […]
Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=synchronizeBasicCustomerJob at state=synchronizeBasicCustomerJob.writeJobIntoJobContextStep0 with exception
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    ... 41 more
Caused by: java.lang.IllegalArgumentException: Could not serialize the execution context
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.serializeContext(JdbcExecutionContextDao.java:306) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.updateExecutionContext(JdbcExecutionContextDao.java:146) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext(SimpleJobRepository.java:216) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_202]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_202]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295) ~[spring-tx-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.10.RELEASE.jar:5.1.10.RELEASE]
    at com.sun.proxy.$Proxy242.updateExecutionContext(Unknown Source) ~[?:?]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:159) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) ~[spring-batch-core-4.1.2.RELEASE.jar:4.1.2.RELEASE]
    ... 41 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Infinite recursion (StackOverflowError) (through reference chain: org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->java.util.Collections$UnmodifiableRandomAccessList[0]->org.springframework.batch.core.StepExecution["jobExecution"]->org.springframework.batch.core.JobExecution["stepExecutions"]->…

The cause is pretty straightforward: The entity JobExecution has a list of StepExecutions and StepExecution has a backreference to JobExecution, which is cyclic, which causes Jackson to throw this ugly error.

The recommended solution to this problem seems to be to annotate the classes to be serialized with Jackson annotations (@JsonManagedReference and @JsonBackReference). But that is not possible in this context, as the offending classes are Spring Batch's.

Another solution is to configure another ExecutionContextSerializer, i.e. the XStreamExecutionContextStringSerializer. This solution works for me, but the XStreamExecutionContextStringSerializer class is deprecated and I'd rather not use that one.

Other people (Migration to Spring Boot 2 and using Spring Batch 4) have the reverse problem: In their case the deserialization of the persisted execution context fails because it was written by XStream and now can not be read by Jackson. But in my case, the serialization fails, so this is not a compatibility issue between XStream and Jackson.

What am I doing wrong?

Answer 1

As I stated in the comments section, you can use Jackson MixIns to address your issue since you don't have access to the spring batch source code in your project.

It would work like

public abstract class JobExecutionMixin {
  @JsonManagedReference
  private Collection<StepExecution> stepExecutions;
}

And for the back reference:

public abstract class StepExecutionsMixin {
  @JsonBackReference
  private final JobExecution jobExecution;
}

And then, you will have to implement the ObjectMapper bean in your spring boot app with something like:

ObjectMapper mapper = new ObjectMapper();
mapper.addMixInAnnotations(org.springframework.batch.core.StepExecutions.class, StepExecutionsMixin.class);
mapper.addMixInAnnotations(org.springframework.batch.core.JobExecution.class, JobExecutionMixin.class);