Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring-Batch without persisting metadata to database?

I want to create a spring-batch job, but I want to run it without any database persistence. Unfortunately spring-batch requires to write metadata ob the job cycles to a database somehow, thus procing me to provide at least some kind of db with transactionmanager and entitymanager.

It it possible to prevent the metadata and run independently from txmanagers and databases?

Update:

ERROR org.springframework.batch.core.job.AbstractJob: Encountered fatal error executing job java.lang.NullPointerException     at org.springframework.batch.core.repository.dao.MapJobExecutionDao.synchronizeStatus(MapJobExecutionDao.java:158) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:161) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]     at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]     at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at com.sun.proxy.$Proxy134.update(Unknown Source) ~[?:?]     at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:416) ~[spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:299) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_51]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_51]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_51]     at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_51]     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.1.RELEASE.jar:3.0.1.RELEASE]     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) [spring-aop-4.0.6.RELEASE.jar:4.0.6.RELEASE]     at com.sun.proxy.$Proxy50.run(Unknown Source) [?:?] 
like image 524
membersound Avatar asked Aug 01 '14 09:08

membersound


People also ask

Does Spring Batch need a database?

Spring Batch by default uses a database to store metadata on the configured batch jobs. In this example, we will run Spring Batch without a database. Instead, an in-memory Map based repository is used.

Will Spring Batch creates it own metadata tables?

The meta table's scripts are stored in the spring-batch. jar , you need to create it manually. Run your Spring batch jobs again, those meta tables will be created automatically.

How do I disable Spring Batch meta data table calls?

1 Answer. Show activity on this post. When you use @EnableBatchProcessing , by default Spring Batch will use the datasource you defined in your application context for its meta-data. If you want to use the map based job repository , you need to implement BatchConfigurer and override getJobRepositoy .

Can we skip processor in Spring Batch?

Using skip and skipLimit. First of all, to enable skip functionality, we need to include a call to faultTolerant() during the step-building process. Within skip() and skipLimit(), we define the exceptions we want to skip and the maximum number of skipped items.


2 Answers

Simply create a configuration without datasource for Batch configuration :

@Configuration @EnableAutoConfiguration @EnableBatchProcessing public class BatchConfiguration extends DefaultBatchConfigurer {      @Override     public void setDataSource(DataSource dataSource) {         // override to do not set datasource even if a datasource exist.         // initialize will use a Map based JobRepository (instead of database)     }  } 

It will initialize JobRepository and JobExplorer with a in memory map based implementation. https://github.com/spring-projects/spring-batch/blob/342d27bc1ed83312bdcd9c0cb30510f4c469e47d/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/DefaultBatchConfigurer.java#L84

and you can use you're production datasource as well even if auto configured with spring boot.

like image 200
Aure77 Avatar answered Oct 09 '22 11:10

Aure77


I want to run it without any database persistence

You can use MapJobRepositoryFactoryBean and ResourcelessTransactionManager

sample configuration:

<bean id="transactionManager"     class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />  <bean id="jobRepository"     class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">     <property name="transactionManager" ref="transactionManager" /> </bean>  <bean id="jobLauncher"     class="org.springframework.batch.core.launch.support.SimpleJobLauncher">     <property name="jobRepository" ref="jobRepository" /> </bean> 

For Spring 4.X, the annotation based configuration would be as follows:

@Bean public PlatformTransactionManager getTransactionManager() {     return new ResourcelessTransactionManager(); }  @Bean public JobRepository getJobRepo() {     return new MapJobRepositoryFactoryBean(getTransactionManager()).getObject(); } 
like image 20
Braj Avatar answered Oct 09 '22 11:10

Braj