Keyword search engine that returns statistics instead of hits

Question

First post on StackOverflow, but I've always looked to this site as a great source of shared knowledge, and I'm excited to see what comes up from this question.

As I feel I have now reached the limits of what I can do with SQL indexes, statistics and full-text search, I'm currently looking for a search library that can provide us with the functionality we need. I'm not averse to writing it myself (and open-sourcing it if I can get the boss's approval), but I would prefer to find something open-source that already exists, natch.

What we're after is a search engine that can provide statistics about the results that are matched when a user searches for a specific keyword. Let's say, for example, that we were talking about a database of products in an online shop. We need to be able to return statistics about how many products there are that match a given set of keywords (and also be able to filter this result set by price, category, etc), as well as the total number of products in stock (assuming that this is stored in a field in the product table). All the search engines that I have found return the top n results, and if you want statistics about the size of the result set, you need to enumerate the whole set. Even if you didn't you still would need to do so to retrieve the total number of products in stock.

Is there anything anyone knows of that is capable of this functionality? As I say, I'm happy to get my hands dirty and either build it myself, or modify the functionality of something like Lucene, but I have not been able to find anything appropriate on Google.

Thanks in advance guys!

Answer 1

You might take a look at Solr, which is a faceted search engine built on top of Lucene. Solr will count lots of different things for you, in addition to doing full-text search. It is good at handling combinations of structured and full-text data.