I am facing weird issue with an application which is a spring boot application.
Details Here:
The app has a spring batch job which uses JpaItemWriter for performing the writes to the database.
Also, the application is configured to use hibernate
ItemWriter configuration as follows:
@Bean(name = "itemWriter")
@StepScope
public ItemWriter<Record> itemWriter() {
JpaItemWriter<Record> itemWriter = new JpaItemWriter<>();
itemWriter.setEntityManagerFactory(emf);
return itemWriter;
}
The batch job used to work great, but recently we upgraded our Jenkins to Jenkins(2.73.3). After that, the code built and deployed using this new Jenkins started throwing the following error when the same batch job is run:
2017-12-03 16:09:44.720 DEBUG - testservice - - jobLauncher-3 - org.springframework.batch.item.database.JpaItemWriter:97 - Writing to JPA with 4 items.
2017-12-03 16:09:44.733 DEBUG - testservice - - jobLauncher-3 - org.springframework.batch.item.database.JpaItemWriter:109 - 4 entities merged.
2017-12-03 16:09:44.733 DEBUG - testservice - - jobLauncher-3 - org.springframework.batch.item.database.JpaItemWriter:110 - 0 entities found in persistence context.
2017-12-03 16:09:44.743 WARN - testservice - - jobLauncher-3 - com.shared.domain.error.BatchExceptionHandler:63 - [test-service]999999999(Dn9lM4c)ExhaustedRetryException: Unable to locate ErrorDetail for Id ExhaustedRetryException. Details:
org.springframework.retry.ExhaustedRetryException: Retry exhausted after last attempt in recovery path, but exception is not skippable.; nested exception is javax.persistence.TransactionRequiredException: no transaction is in progress
at org.springframework.batch.core.step.item.FaultTolerantChunkProcessor$5.recover(FaultTolerantChunkProcessor.java:403) ~[spring-batch-core-3.0.7.RELEASE.jar!/:3.0.7.RELEASE]
at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:473) ~[spring-retry-1.1.5.RELEASE.jar!/:?]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:333) ~[spring-retry-1.1.5.RELEASE.jar!/:?]
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:201) ~[spring-retry-1.1.5.RELEASE.jar!/:?]
The same code base when built and deployed using Old Jenkins (Jenkins ver. 1.646), the job works fine.
In both cases, the application is getting deployed on an EC2 instance ( AWS ) with same AMI. So there is no difference in the server which they are running on.
Also, I checked the Java version and maven version used for building the code on both versions of Jenkins and they are same.
Maven version and Java version:
Apache Maven 3.0.4 (r1232337; 2012-01-17 08:44:56+0000)
Maven home: /opt/maven/apache-maven-3.0.4
Java version: 1.8.0_72, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_72/jre
Default locale: en_US, platform encoding: UTF-8
Other versions:
SpringBoot: 1.4.7.RELEASE
HibernateVersion: 5.2.3.Final
The only difference is the old Jenkins is running on AWS RHEL and the new one is running on CentOS7.
I even checked the jars created in both the cases and nothing seems to be different.
Currently not sure how to further troubleshoot this issue
Edited:
I did remote debugging of the code generated by both the Jenkins, the following is the observations made.
JpaItemWriter calls "EntityManagerFactoryUtils.getTransactionalEntityManager" and in the method "emHolder" is null in one case and has valid value in the other case.
public static EntityManager doGetTransactionalEntityManager(
EntityManagerFactory emf, Map<?, ?> properties, boolean synchronizedWithTransaction) throws PersistenceException {
Assert.notNull(emf, "No EntityManagerFactory specified");
EntityManagerHolder emHolder =
(EntityManagerHolder) TransactionSynchronizationManager.getResource(emf);
// This line returns a valid emHolder
// But in case of the code which is not working, it is returning a
// null value
EntityManagerHolder emHolder =
(EntityManagerHolder) TransactionSynchronizationManager.getResource(emf);
Not sure why the NamedThreadLocal>("Transactional resources") does not have the entityMangerHolder in one case but not in other case.
Update
Upon more debugging what I found in the case of code which is not working spring batch job uses "DataSourceTransactionManager" but in other case it uses the JpaTransactionManager.
I was able to solve my problem for now, here are my findings:
Seems like there is a difference in the order in which the beans get created in both the cases.
I had a JpaTransaction manager bean defined in of my configuration classes.This bean is never created with the code that was generated using the Jenkins 2.0.
Spring is generating a proxy transaction manager bean in the "org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration" class and this transaction manager is not working well with the JpaItemWriter.
I had seen a JIRA ticket which discusses the problem with SimpleBatchConfiguration.
https://jira.spring.io/browse/BATCH-2294
After I added the same JpaTransaction bean in my BatchConfiguration class, the problem disappeared.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With