Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

solr Data Import Handlers for MongoDB

I am working on a project where we have millions of entries stored in MongoDB database and, i want to index all this data using SOLR.

After extensive Searching i came to know there are no proper "Data Import Handlers" for mongoDB database.

Can anyone tell me what are the proper approaches for indexing data in MongoDB using SOLR ?

I want to use all the features of SOLR and want it to be scalable in real-time. I saw one or two approaches from different posts but not sure how they will work real time..

Many Thanks

like image 651
kich Avatar asked Feb 18 '12 23:02

kich


3 Answers

10Gen introduce Mongodb Connector. You can integrate Mongodb with Solr using this tool.

Blog post : Introducing Mongo Connector

Github page : mongo-connector

like image 92
Parvin Gasimzade Avatar answered Nov 18 '22 10:11

Parvin Gasimzade


I have created a plugin to allow you to load data from MongoDb using the Solr data import handler.

Check it out at:

https://github.com/james75/SolrMongoImporter

like image 26
user1607179 Avatar answered Nov 18 '22 10:11

user1607179


I wrote a response to a similar question, except it was how to import data from MySQL into SOLR. The example code is in PHP, but should give you a general idea. All you would need to do is set up an iterator to step through your MongoDB assets, extract the data to SOLR datatypes, and then save it to your SOLR index.

If you want it to be real-time, you could add some custom code to the save mechanism (assuming this can be done with MongoDB), and save directly to the SOLR index, then run a commit script to commit data every 15 minutes (via cron).

like image 5
Mike Purcell Avatar answered Nov 18 '22 09:11

Mike Purcell