Deserializing Elasticsearch Results via JSON.NET

Question

I have a .NET application that I want to use to query Elasticsearch from. I am successfully querying my Elasticsearch index. The result looks similar to this:

{
  "took":31,
  "timed_out":false,
  "_shards": {
    "total":91,
    "successful":91,
    "skipped":0,
    "failed":0
  },
  "hits":{
    "total":1,
    "max_score":1.0,
    "hits":[
      {
        "_index":"my-index",
        "_type":"doc",
        "_id":"TrxrZGYQRaDom5XaZp23",
        "_score":1.0,
        "_source":{
          "my_id":"65a107ed-7325-342d-adab-21fec0a97858",
          "host":"something",
          "zip":"12345"
        }
      },
    ]
  }
}

Right now, this data is available via the Body property on the StringResponse I'm getting back from Elasticsearch. I want to deserialize the actual records (I don't want or need the took, timed_out, etc. properties) into a C# object named results. In an attempt to do this, I have:

var results = JsonConvert.DeserializeObject<List<Result>>(response.Body);

The Result class looks like this:

public class Result
{
  [JsonProperty(PropertyName = "my_id")]
  public string Id { get; set; }

  [JsonProperty(PropertyName = "host")]
  public string Host { get; set; }

  [JsonProperty(PropertyName = "zip")]
  public string PostalCode { get; set; }
}

When I run this, I get the following error:

Cannot deserialize the current JSON object into type 'System.Collections.Generic.List`1[Result]' because the type requires a JSON array to deserialize correctly.

While the error makes sense, I don't know how to parse the hits to just extract the _source data. The _source property contains the data I want to deserialize. Everything else is just metadata that I don't care about.

Is there a way to do this? If so, how?

Answer 1

You can use Json.Net's LINQ-to-JSON API to get just the nodes you are interested in and then convert those to a list of results:

var results = JToken.Parse(response.Body)
                    .SelectTokens("hits.hits[*]._source")
                    .Select(t => t.ToObject<Result>())
                    .ToList();

Working demo: https://dotnetfiddle.net/OkEpPA

Answer 2

Well you're DeserializeObject<T> T does not match the Json. Your Json starts with a { so your T needs to be an class (not an IEnumerable type).

Let's start outside and work our way in:

{
  "took":31,
  "timed_out":false,
  "_shards": <object>
  "hits": <object>
}

so:

public class SearchResult
{
  [JsonProperty("took")]
  public int Took { get; set; }
  [JsonProperty("timed_out")]
  public bool TimedOut { get; set; }
  [JsonProperty("_shards")]
  public Shards Shards { get; set; }
  [JsonProperty("hits")]
  public Hits Hits { get; set; }
}

next is _shards

"_shards": {
  "total":91,
  "successful":91,
  "skipped":0,
  "failed":0
},

so

public class Shards 
{
  [JsonProperty("total")]
  public int Total { get; set; }
  // etc...
}

Then hits

{
  "total":1,
  "max_score":1.0,
  "hits": <IEnumerable because []>
}

so

public class Hits
{
  [JsonProperty("total")]
  public int Total { get; set; }
  [JsonProperty("max_score")]
  public int MaxScore { get; set; }
  [JsonProperty("hits")]
  public List<Hit> Hits { get; set; }
}

then Hits list:

{
    "_index":"my-index",
    "_type":"doc",
    "_id":"TrxrZGYQRaDom5XaZp23",
    "_score":1.0,
    "_source":  <object>
},

so

public class Hit
{
  [JsonProperty("_index")]
  public string Index { get; set; }
  // etc
}

And once you've created all of them you need, then you deserialize:

JsonConvert.DeserializeObject<SearchResult>(json);

Answer 3

You will need to first deserialize to a generic JToken or JObject, like this:

var token = JsonConvert.DeserializeObject<JToken>(jsonString);

And then you can navigate to the _source property that holds the data of your interest:

var hitsArray = token["hits"]["hits"] as JArray;
var result = hitsArray[0]["_source"].ToObject<Result>();

Answer 4

i used http://json2csharp.com/ to convert a json to c# classes and for my test i got a json string from conversion made on http://easyonlineconverter.com/converters/dot-net-string-escape.html

then i have created a console app with this class:

using System.Collections.Generic;
using Newtonsoft.Json;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            string json = "{  \"took\":31,  \"timed_out\":false,  \"_shards\": {    \"total\":91,    \"successful\":91,    \"skipped\":0,    \"failed\":0  },  \"hits\":{    \"total\":1,    \"max_score\":1.0,    \"hits\":[      {        \"_index\":\"my-index\",        \"_type\":\"doc\",        \"_id\":\"TrxrZGYQRaDom5XaZp23\",        \"_score\":1.0,        \"_source\":{          \"my_id\":\"65a107ed-7325-342d-adab-21fec0a97858\",          \"host\":\"something\",          \"zip\":\"12345\"        }      },    ]  }}";
            RootObject t = JsonConvert.DeserializeObject<RootObject>(json);
        }

        public class Shards
        {
            public int total { get; set; }
            public int successful { get; set; }
            public int skipped { get; set; }
            public int failed { get; set; }
        }

        public class Source
        {
            public string my_id { get; set; }
            public string host { get; set; }
            public string zip { get; set; }
        }

        public class Hit
        {
            public string _index { get; set; }
            public string _type { get; set; }
            public string _id { get; set; }
            public double _score { get; set; }
            public Source _source { get; set; }
        }

        public class Hits
        {
            public int total { get; set; }
            public double max_score { get; set; }
            public List<Hit> hits { get; set; }
        }

        public class RootObject
        {
            public int took { get; set; }
            public bool timed_out { get; set; }
            public Shards _shards { get; set; }
            public Hits hits { get; set; }
        }
    }
}

hope this helps