With C#, is querying YAML possible without defining lots of types?

Question

I need to work with YAML generated by Kubernetes and I'd like to be able to read specific properties with an XPath-like or jq-like DSL notation in C#.

The structure and nature of the YAML that Kubernetes generates is well-defined in most places, but in some cases is arbitrary and comes from user input, so it's not possible to define static types up front that can capture the entire structure of the YAML.

The most popular solution for deserializing and reading YAML in C# seems to be YamlDotNet, but it's mostly geared towards deserializing into fully-typed objects.

I'd rather not have to define a bunch of static types or do a lot of cumbersome casting just to get one or two fields or aggregate across them. My ideal approach would instead be something like:

var reader = new FileReader("my-file.yaml");
List<string> listOfPodNames = Yaml.Deserialize(reader)
                                  .Query(".pods[*].name")
                                  .AsList;
// expected result: list of all pod names as strings

Is this possible with YamlDotNet or another similar and well-supported tool in C#?

Update: I tried a number of approaches, but in the end, the one that worked best was reserializing to JSON and then querying with Json.NET, which has better support.

Answer 1

When using YamlDotNet Deserializing mechanism without specifying a target type, we always get a either a Dictionary (mapping),a List of KeyValuePairs (list) or a single KeyValuePair/string (scalar). The KeyValuePairs will either contain another Dictionary, another List or the actual value.

We now can implement a query functionality:

var data = new YamlQuery(yamlObject)
                        .On("pods")  // parent
                      // this functionality could be implemented as well wihtout much effort
                      //.Where("ignore").Equals(true)
                        .Get("name") // propery
                        .ToList<string>();

Edit: Multiple nested values

var data = new YamlQuery(yamlObject)
                .On("ressources")
                .On("pods")
                .Get("name")
                .ToList<string>();

Working example: https://dotnetfiddle.net/uNQPyl

using System.IO;
using System;
using System.Linq;
using YamlDotNet.Serialization;
using System.Collections.Generic;
using YamlDotNet.RepresentationModel;

namespace ConsoleApp1
{
    public class Program
    {
        public static void Main()
        {
            object yamlObject;
            using (var r = new StringReader(Program.Document))
                yamlObject = new Deserializer().Deserialize(r);

            var data = new YamlQuery(yamlObject)
                                .On("pods")
                                .Get("name")
                                .ToList<string>();
            Console.WriteLine("all names of pods");
            Console.WriteLine(string.Join(",", data));


            data = new YamlQuery(yamlObject)
                    .On("ressources")
                    .On("pods")
                    .Get("name")
                    .ToList<string>();
            Console.WriteLine("all names of pods in ressources");
            Console.WriteLine(string.Join(",", data));

        }

        public class YamlQuery
        {
            private object yamlDic;
            private string key;
            private object current;

            public YamlQuery(object yamlDic)
            {
                this.yamlDic = yamlDic;
            }

            public YamlQuery On(string key)
            {
                this.key = key;
                this.current = query<object>(this.current ?? this.yamlDic, this.key, null);
                return this;
            }
            public YamlQuery Get(string prop)
            {
                if (this.current == null)
                    throw new InvalidOperationException();

                this.current = query<object>(this.current, null, prop, this.key);
                return this;
            }

            public List<T> ToList<T>()
            {
                if (this.current == null)
                    throw new InvalidOperationException();

                return (this.current as List<object>).Cast<T>().ToList();
            }

            private IEnumerable<T> query<T>(object _dic, string key, string prop, string fromKey = null)
            {
                var result = new List<T>();
                if (_dic == null)
                    return result;
                if (typeof(IDictionary<object, object>).IsAssignableFrom(_dic.GetType()))
                {
                    var dic = (IDictionary<object, object>)_dic;
                    var d = dic.Cast<KeyValuePair<object, object>>();

                    foreach (var dd in d)
                    {
                        if (dd.Key as string == key)
                        {
                            if (prop == null)
                            { 
                                result.Add((T)dd.Value);
                            } else
                            { 
                                result.AddRange(query<T>(dd.Value, key, prop, dd.Key as string));
                            }
                        }
                        else if (fromKey == key && dd.Key as string == prop)
                        { 
                            result.Add((T)dd.Value);
                        }
                        else
                        { 
                            result.AddRange(query<T>(dd.Value, key, prop, dd.Key as string));
                        }
                    }
                }
                else if (typeof(IEnumerable<object>).IsAssignableFrom(_dic.GetType()))
                {
                    var t = (IEnumerable<object>)_dic;
                    foreach (var tt in t)
                    {
                        result.AddRange(query<T>(tt, key, prop, key));
                    }

                }
                return result;
            }
        }




        private const string Document = @"---
            receipt:    Oz-Ware Purchase Invoice
            date:        2007-08-06
            customer:
                given:   Dorothy
                family:  Gale

            pods:
                - name:   pod1
                  descrip:   Water Bucket (Filled)
                  price:     1.47
                  quantity:  4


                - name:   pod2
                  descrip:   High Heeled ""Ruby"" Slippers
                  price:     100.27
                  quantity:  1
                - name:   pod3
                  descrip:   High Heeled ""Ruby"" Slippers
                  ignore:    true
                  quantity:  1

            bill-to:  &id001
                street: |-
                        123 Tornado Alley
                        Suite 16
                city:   East Westville
                state:  KS
                pods:
                    - name: pod4
                      descrip:   High Heeled ""Ruby"" Slippers
                      price:     100.27
                      quantity:  
            ressources:
                      - pids:
                            - id: 1
                            - name: pid
                      - pods: 
                            - name: pod5
                              descrip:   High Heeled ""Ruby"" Slippers
                              price:     100.27
                              quantity:  
                            - name: pod6
                              descrip:   High Heeled ""Ruby"" Slippers
                              price:     100.27
                              quantity:  
            specialDelivery: >
                Follow the Yellow Brick
                Road to the Emerald City.
                Pay no attention to the
                man behind the curtain.

...";
    }

}

Answer 2

Another approach you can use is converting YAML to JSON and then query it. though it would be a more time-consuming approach you can surely query a JSON easily then YAML.

Here is how you can do it

Convert YAML To JSON

    public class ConvertYamlToJson
    {
        private readonly ITestOutputHelper output;

        public ConvertYamlToJson(ITestOutputHelper output)
        {
            this.output = output;
        }

        [Sample(
            DisplayName = "Convert YAML to JSON",
            Description = "Shows how to convert a YAML document to JSON."
        )]
        public void Main()
        {
            // convert string/file to YAML object
            var r = new StringReader(@"
scalar: a scalar
sequence:
  - one
  - two
");
            var deserializer = new DeserializerBuilder().Build();
            var yamlObject = deserializer.Deserialize(r);

            var serializer = new SerializerBuilder()
                .JsonCompatible()
                .Build();

            var json = serializer.Serialize(yamlObject);

            output.WriteLine(json);
        }
    }

Ref:- Convert YAML to JSON

Query JSON

 string json = @"
            {
              ""client_id"": ""26075235"",
              ""client_version"": ""1.0.0"",
              ""event"": ""app.uninstall"",
              ""timestamp"": 1478741247,
              ""data"": {
                ""user_id"": ""62581379"",
                ""site_id"": ""837771289247593785"",
                ""platform_app_id"": ""26075235""
              }
            }";

        JObject jo = JObject.Parse(json);

        Console.WriteLine("User ID: " + (string)jo.SelectToken("data.user_id"));

Ref:- JSON.NET JObject - how do I get value from this nested JSON structure

Answer 3

There is the following GitHub project: YamlDotNet.Dynamic

It leverages the dynamic type in C# and therefore you don't need to define static types.

---

A different approach would be to convert into Json and use Newtonsoft.Json, which supports dynamic types, too.