Can Spring batch used for data processing or it is only an ETL tool?

Question

I'm trying to utilize Spring Batch in one of the projects that I have, as there is another project that is based on Spring Batch.

However the more I read the more I realize that Spring batch is nothing like ApacheBeam or MapReduce, it is only used for transferring the SAME data from one place to another with some type mapping like varchar -> string.

However, the task in hand requires some processing, not only types mapping and converting but also aggregations and data structure. Can Spring batch used for data processing or it is only an ETL tool ?

Answer 1

well, i disagree on this point that spring batch - is only used for transferring the SAME data from one place to another with some type mapping like varchar -> string.

Worked in 4 years in this technology and have witnessed this framework grow a lot.

Spring batch is well capable of processing data, mapping, required conversion and data aggregations - spring batch can definitely be used for data processing .

being open source technology - you will get lot of material to read about, and the forums like stackoverflow have ton of FAQs around it.

For scaling and paralleling there are various architectures in spring batch, which will help in enhancing your performance.

Further details you can find here

SPRING_BATCH_SCALING_AND_PARALLELING

If you want to monitor your jobs then you cas use - Spring cloud date flow. Monitoring can also be done - with AppDynamics.

Referrer this blog -

MONITOR_SPRING_BATCH_JOB_WITH_APP_DYNAMICS

Another advantage of using spring batch is you have lot of standerd predefined reader , processor and writer types - which support sources like file , DB , stream etc..

On top of this - as it is a java based framework you can do all stuff that can be done with java.

I hope this helps.