Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Full text search options for MongoDB setup

We are planning to store millions of documents in MongoDB and full text search is very much required. I read Elasticsearch and Solr are the best available solutions for full text search.

  • Is Elastic search is mature enough to be used for Mongodb full text search? We also be sharding the collections. Does Elasticsearch works with Sharded collections?

  • What are the advantages and disadvantages of using Elasticsearch or Solr?

  • Is MongoDB capable of doing full text search?

like image 224
atandon Avatar asked Jun 13 '12 13:06

atandon


People also ask

How do I search for text in a field in MongoDB?

In MongoDB, we can perform text search using text index and $text operator. Text index: MongoDB proved text indexes that are used to find the specified text from the string content. Text indexes should be either a string or an array of string elements.

Does MongoDB provide a facility to do text searches How?

For on-premises (non-Atlas) deployments, MongoDB's text search capability supports query operations that perform a text search of string content. To perform text search, MongoDB uses a text index and the $text operator.

How does text search work in MongoDB?

MongoDB text search uses the Snowball stemming library to reduce words to an expected root form (or stem) based on common language rules. Algorithmic stemming provides a quick reduction, but languages have exceptions (such as irregular or contradicting verb conjugation patterns) that can affect accuracy.


2 Answers

There are some search capabilities in MongoDB but it is not as feature-rich as search engines.

http://www.mongodb.org/display/DOCS/Full+Text+Search+in+Mongo

We use Mongo with Solr to make content searchable. We prefer Solr because

  • It is easy to configure and customize
  • It has large community (This is really helpful if you are working with opensource tools)

Since we didn't work with ES i could not say much about it. You can found some discussions about Solr vs ES on the links below.

  • Solr vs ES 1
  • Solr vs ES 2
  • Solr vs ES 3
like image 191
Parvin Gasimzade Avatar answered Sep 25 '22 23:09

Parvin Gasimzade


I have a professional experience with both Solr/MySQL and ElasticSearch/MongoDB.

If you are going to query a lot your search engine, you already shard your MongoDB (I mean, if you want to shard too your search engine): you should use ElasticSearch, unless what you want to do can't be done with ElasticSearch. And you should use it even if you are not going to shard.

ElasticSearch is a new project on top of Lucene that brings the sharding mechanism, from someone who is used to distributed environments and search (Shay Bannon made Compass and worked for Gigaspaces, the datagrid editor).

ElasticSearch is as easy as MongoDB to shard, I think it is even simpler and the default works great for most cases.


I don't like Solr so much.

  • The query langage is not structured at all (but it's the case of plugins and Lucene, and I think you can use this unstructured query langage with ES too)
  • I don't think there is a proper Solr client. Solr java client sucks, and I hearh PHP guys also complaining, while ElasticSearch Java client is very nice, much more typesafe and offers async support (nice if you use Netty for exemple). With Solr, you will do a LOT of string concatenation.
  • Less easy to scale
  • Not so new project, I felt the technical dept it has. ElasticSearch is born from Compass, so I guess all the technical dept has been dropped to have a fresh new approach.

Concerning data importing, I have experience with both Solr DataImportHandler and ElasticSearch rivers (CouchDB and MongoDB). What I can tell you is:

  • Solr permits to do more things, but in a very unstructured XML way, and the documentation doesn't help you so much to understand what is really happing once you are out of the hello world and try to use some advanced features.
  • ElasticSearch approach is more simple and also limited but has out of the box support for some technologies while DataImportHandler seems more complex-SQL friendly
  • With my Solr project I had to use manual indexation for some documents, but it was mostly because of the impossibility to denormalize the needed data into a document (the Solr project uses MySQL).

There is also a new MongoDB connector for both Solr and ElasticSearch which I need to test asap :) http://blog.mongodb.org/post/29127828146/introducing-mongo-connector


So in the end, I'll definitly choose ElasticSearch, because:

  • It now has a great community
  • Many people I know with experience with Solr like ElasticSearch
  • The client side is safer and structured, and provides async with Java Futures
  • Both can probably import data from MongoDB easily with the new connector
  • As far as I know, it permits to do almost everything Solr does (in my experience but I'm not a search engine expert)
  • It adds sharding out of the box
  • It adds percolation which can help to built realtime scalable applications (but you'll probably need an additional messaging technology)
  • The source code I read has nearly no technical dept compared to Solr (at least on the client side), and it seems easy to create plugins.
like image 36
Sebastien Lorber Avatar answered Sep 25 '22 23:09

Sebastien Lorber