Use LINQ to Process Elastic Search Results

Question

What is the most most-efficient method for using LINQ to process ElasticSearch Results?

I came across JSON.Net's JObject Class.

Is the JSON returned from ElasticSearch structured in a fashion that lends itself to proper LINQ Queries via the JObject Class?

Answer 1

This is a complete solution on how to utilize LINQ to process the JSON query-results from the elasticsearch engine.

Connection Library - PlainElastic.NET

First off, I am using PlainElastic.NET for one single purpose: for connecting to my elasticsearch server. This library is very lean and contains concise GET, POST, and PUT functions. This comes into play below with the client connection object. result.ToString() provides the JSON response from the elasticsearch server. Just slap the DLL into your bin and add a reference.

JSON Processor - JSON.NET

I am using a great feature of NewtonSoft JSON.NET to facilitate LINQ queries. The library has a class JObject which provides a nestable, queryable structure to execute LINQ against. See the line below where I grab HitCount to see how nice it is to grab an individual value, and, of course, check out the LINQ query to see how to query against it. Remember, this is simply a data-structure that is the result of parsing a JSON string. Just slap the DLL into your bin and add a reference.

Performing the Query against Elastic Search

'Execute the Search
Dim client = New ElasticConnection("localhost", 9200)
Dim result = client.Post("myindex/mytype/_search", elastic_query) 'elastic_query = well-formatted elasticsearch query (json string)'
Dim J As JObject = JObject.Parse(result.ToString())

'How many results were located? 
Dim HitCount = CInt(J("hits")("total"))

'What was the maximum score? Maybe you want to know this for normalizing scores to 0-100.
' - We make sure we have hits first
' - Also, make sure scoring is turned on. It might be turned off if an alternate sort method is used in your query
If HitCount > 0 AndAlso J("hits")("max_score").Type <> JTokenType.Null Then MaxScore = CDbl(J("hits")("max_score"))

Processing the Results with LINQ

Now, use LINQ to provide a beautiful anonymous object for binding to Gridviews, DataLists, Repeaters, etc. I am intentionally providing an example of a non-trivial LINQ query. This example gets nested data (Stores, Categories)

Dim SearchResults = _
  (From X In J("hits")("hits")
    Select
      score = CDec(If(X("_score").Type = JTokenType.Null, 0, X("_score"))),
      boost = CDbl(X("_source")("_boost")),
      InstitutionID = CInt(X("_source")("InstitutionID")),
      Name = CStr(X("_source")("Name")),
      Stores = (From S In X("_source")("Stores") Select CInt(S("StoreID"))).ToList(),
      Categories = (From Z In X("_source")("Categories") Select CatID = CInt(Z("CatID")), CatName = CStr(Z("CatName")), InstitutionID = CInt(X("_source")("InstitutionID"))).ToList()
 Order By score Descending, boost Descending
)

Now, you can bind to a Repeater, DataList, or Gridview

MyRepeater.DataSource = SearchResults
MyRepeater.DataBind()