How to search over huge non-text based data sets?

Question

In a project I am working, the client has a an old and massive(terabyte range) RDBMS. Queries of all kinds are slow and there is no time to fix/refactor the schema. I've identified the sets of common queries that need to be optimized. This set is divided in two: full-text and metadata queries.

My plan is to extract the data from their database and partition it across two different storage systems each optimized for a particular query set.

For full-text search, Solr is the engine that makes most sense. It's sharding and replication features make it a great fit for half of the problem.

For metadata queries, I am not sure what route to take. Currently, I'm thinking of using an RDBMS with an extremely de-normalized schema that represents a particular subset of the data from the "Authoritative" RDBMS. However, my client is concerned about the lack of sharding and replication of such subsystem and difficulty/complications of setting such features as compared with Solr that already includes them. Metadata in this case takes the form of integers, dates, bools, bits, and strings(with max size of 10chars).

Is there a database storage system that features built-in sharding and replication that may be particular useful to query said metadata? Maybe a no-sql solution out there that provides a good query engine?

Illuminate please.

Additions/Responses:

Solr can be used for metadata, however, the metadata is volatile. Therefore, I would have to commit often to the indexes. This would cause search to degrade pretty fast.

Answer 1

RavenDB:

• It has Lucene built-in for your full-text searches.
• It can be replicated.
• It supports sharding.
• It has a HTTP API so you can in principle use it from any platform.

Cons: it's AGPL licensed. Depending on your dev/server environment, you could consider it running on .NET a con. Also I'm not aware of the status of clients for other plaforms than .NET.

Solandra:

• Integrates Solr and Cassandra
• Full-text search managed by Solr
• Replication and sharding managed by Cassandra

Cons: not yet released.

ElasticSearch:

• It has Lucene built-in for your full-text searches.
• It can be replicated.
• It supports sharding.
• It has a HTTP API so you can in principle use it from any platform.

ElasticSearch looks similar to RavenDB but it seems to emphasize full-text search where RavenDB emphasizes being a general NoSQL database.