Parallel ordered processing of spring integration events

Question

am pretty new to the integration framework. I have a spring message listener which receives updates for stocks ( S1, S2, S3 ...). The updates for same stock need to be processed sequentially where as updates for different stocks should be processed parallel.

e.g. if sequence of updates are S1-1, S1-2, S2-1, S1-3, S3-1, S2-2, S1-3, S3-2 .. then there should be three parallel stream of processing

S1-1, S1-2, S1-3

S2-1, S2-1

S3-1, S3-2

Note that there could be thousands of such stocks.

Currently I am processing everything in parallel using executor on the channel. how can i achieve my requirements. please advise. thanks.

Answer 1

Well, since we can't parallel "thousands of such stocks" because spawning so many threads is really inefficient, so I would go something like partitioning and queue channels for each of them.

The router component could decide by the request message which partition it belongs and send to respective queue channel.

This way a poller on that queue channel would pull messages sequentially, but different queues would be processed in parallel.

See more info in docs:

https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#router

https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#channel-implementations-queuechannel