Lucene-like searching through JSON objects in JavaScript

Question

I have a pretty big array of JSON objects (its a music library with properties like artist, album etc, feeding a jqgrid with loadonce=true) and I want to implement lucene-like (google-like) query through whole set - but locally, i.e. in the browser, without communication with web server. Are there any javascript frameworks that will help me?

Answer 1

1. Go through your records, to create a one time index by combining all search able fields in a single string field called index.

2. Store these indexed records in an Array.

3. Partition the Array on index .. like all a's in one array and so on.

4. Use the javascript function indexOf() against the index to match the query entered by the user and find records from the partitioned Array.

That was the easy part but, it will support all simple queries in a very efficient manner because the index does not have to be re-created for every query and indexOf operation is very efficient. I have used it for searching up to 2000 records. I used a pre-sorted Array. Actually, that's how Gmail and yahoo mail work. They store your contacts on browser in a pre-sorted array with an index that allows you to see the contact names as you type.

This also gives you a base to build on. Now you can write an advanced query parsing logic on top of it. For example, to support a few simple conditional keywords like - AND OR NOT, will take about 20-30 lines of custom JavaScript code. Or you can find a JS library that will do the parsing for you the way Lucene does.

For a reference implementation of above logic, take a look at how ZmContactList.js sorts and searches the contacts for autocomplete.

Answer 2

You might want to check FullProof, it does exactly that: https://github.com/reyesr/fullproof