Asynchronous Search

Question

I am currently working on building a proof of concept search solution for my company using Lucene and Hibernate Search. I have built individual components which work fine. I am now looking at creating a single API that would allow a user to get search results back from different sources (domain + data). What I would like to achieve is something like a search manager fires search requests to different search components asynchronously and when one set of results have been processed return that result to the user while processing the rest. Once the result of the search has been processed notify the client that there are more search results available.

I am wondering whether I have a search manager which creates separate threads to search individual search components and keeps a list of search results. Once the list is populated with one set return that back to the user. Any additional search results added would involve the search manager pushing the results to the user.

I am not looking for any code example (any would be appreciated) but I was wondering if I could get some guidance on how to tackle this problem. Do I use event processing technologies (GigaSpaces, Spring, JMS) or use standard Java concurrent libraries. What would be the effective way of managing the list and push the updated results.

Cheers

Answer 1

This sounds like a perfect fit for the Executor Service abstraction in Java 5 and higher. You can submit tasks to a pool of executor threads and asynchronously poll for completion.

So in your case, you'd create each search as it's own task, and then poll those tasks for completion. Once they're done, grab the results and aggregate them for the user.