.Net Core Regular Expressions, Named groups, nested groups, backreferences and lazy qualifier

Question

I'm attempting to parse key-value pairs from strings which look suspiciously like markup using .Net Core 2.1.

Considering the sample Program.cs file below...

My Questions Are:

1.

How can I write the pattern kvp to behave as "Key and Value if exists" instead of "Key or Value" as it currently behaves?

For example, in the test case 2 output, instead of:

=============================
input = <tag KEY1="vAl1">

--------------------
kvp[0] = KEY1
          key   =       KEY1
        value   =
--------------------
kvp[1] = vAl1
          key   =
        value   =       vAl1
=============================

I want to see:

=============================
input = <tag KEY1="vAl1">

--------------------
kvp[0] = KEY1="vAl1"
          key   =       KEY1
        value   =       vAl1
=============================

Without breaking test case 9:

=============================
input = <tag noValue1 noValue2>

--------------------
kvp[0] = noValue1
          key   =       noValue1
        value   =
--------------------
kvp[1] = noValue2
          key   =       noValue2
        value   =
=============================

2.

How can I write the pattern value to stop matching at the next character matched by the group named "quotes"? In other words, the very next balancing quote. I obviously misunderstand how backreferencing works, my understanding is that \k<quotes> will be replaced by the value matched at runtime (not the pattern defined at design time) by (?<quotes>[""'`]).

For example, in the test case 5 output, instead of:

--------------------
kvp[4] =  key3='hello,
          key   =
        value   =        key3='hello,
--------------------
kvp[5] = experts
          key   =
        value   =       experts
=============================

I want to see (notwithstanding a solution to question 1):

--------------------
kvp[4] =  key3
          key   =        key3
        value   =
--------------------
kvp[5] = hello, "experts"
          key   =
        value   =       hello, "experts"
=============================

3.

How can I write the pattern value to stop matching before />? In test case 7, the value for key2 should be thing-1. I can't remember all that I've attempted, but I haven't found a pattern that works without breaking test case 6, in which the / is part of the value.

---

Program.cs

using System;
using System.Reflection;
using System.Text.RegularExpressions;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            RegExTest();

            Console.ReadLine();
        }

        static void RegExTest()
        {
            // Test Cases
            var case1 = @"<tag>";
            var case2 = @"<tag KEY1=""vAl1"">";
            var case3 = @"<tag kEy2='val2'>";
            var case4 = @"<tag key3=`VAL3`>";
            var case5 = @"<tag           key1='val1' 

                        key2=""http://www.w3.org"" key3='hello, ""experts""'>";
            var case6 = @"<tag :key1 =some/thing>";
            var case7 = @"<tag key2=thing-1/>";
            var case8 = @"<tag key3      =        thing-2>";
            var case9 = @"<tag noValue1 noValue2>";
            var case10 = @"<tag/>";
            var case11 = @"<tag />";

            // A key may begin with a letter, underscore or colon, follow by 
            // zero or more of those, or numbers, periods, or dashs.
            string key = @"(?<key>(?<=\s+)[a-z_:][a-z0-9_:\.-]*?(?=[\s=>]+))";

            // A value may contain any character, and must be wrapped in balanced quotes (double, single, 
            // or back) if the value contains any quote, whitespace, equal, or greater- or less- than 
            // character.
            string value = @"(?<value>((?<=(?<quotes>[""'`])).*?(?=\k<quotes>)|(?<=[=][\s]*)[^""'`\s=<>]+))";

            // A key-value pair must contain a key, 
            // a value is optional
            string kvp = $"(?<kvp>{key}|{value})"; // Without the | (pipe), it doesn't match any test case...

            // ...value needs to be optional (case9), tried:
            //kvp = $"(?<kvp>{key}{value}?)";
            //kvp = $"(?<kvp>{key}({value}?))";
            //kvp = $"(?<kvp>{key}({value})?)";
            // ...each only matches key, but also matches value in case8 as key

            Regex getKvps = new Regex(kvp, RegexOptions.IgnoreCase);

            FormatMatches(getKvps.Matches(case1)); // OK

            FormatMatches(getKvps.Matches(case2)); // OK

            FormatMatches(getKvps.Matches(case3)); // OK

            FormatMatches(getKvps.Matches(case4)); // OK

            FormatMatches(getKvps.Matches(case5)); // Backreference and/or lazy qualifier doesn't work.

            FormatMatches(getKvps.Matches(case6)); // OK

            FormatMatches(getKvps.Matches(case7)); // The / is not part of the value.

            FormatMatches(getKvps.Matches(case8)); // OK

            FormatMatches(getKvps.Matches(case9)); // OK

            FormatMatches(getKvps.Matches(case10)); // OK

            FormatMatches(getKvps.Matches(case11)); // OK
        }

        static void FormatMatches(MatchCollection matches)
        {
            Console.WriteLine(new string('=', 78));

            var _input = matches.GetType().GetField("_input",
                BindingFlags.NonPublic |
                BindingFlags.Instance)
                .GetValue(matches);

            Console.WriteLine($"input = {_input}");
            Console.WriteLine();

            if (matches.Count < 1)
            {
                Console.WriteLine("[kvp not matched]");
                return;
            }

            for (int i = 0; i < matches.Count; i++)
            {
                Console.WriteLine(new string('-', 20));

                Console.WriteLine($"kvp[{i}] = {matches[i].Groups["kvp"]}");
                Console.WriteLine($"\t  key\t=\t{matches[i].Groups["key"]}");
                Console.WriteLine($"\tvalue\t=\t{matches[i].Groups["value"]}");
            }
        }
    }
}

Answer 1

You may use

\s(?<key>[a-z_:][a-z0-9_:.-]*)(?:\s*=\s*(?:(?<q>[`'"])(?<value>.*?)\k<q>|(?<value>(?:(?!/>)[^\s`'"<>])+)))?

See the regex demo with groups highlighed and a .NET regex demo (proof).

C# usage:

var pattern = @"\s(?<key>[a-z_:][a-z0-9_:.-]*)(?:\s*=\s*(?:(?<q>[`'""])(?<value>.*?)\k<q>|(?<value>(?:(?!/>)[^\s`'""<>])+)))?";
var matches = Regex.Matches(case, pattern, RegexOptions.IgnoreCase);
foreach (Match m in matches)
{
    Console.WriteLine(m.Value);                 // The whole match
    Console.WriteLine(m.Groups["key"].Value);   // Group "key" value
    Console.WriteLine(m.Groups["value"].Value); // Group "value" value
}

Details

• \s - a whitespace
• (?<key>[a-z_:][a-z0-9_:.-]*) - Group "key": a letter, _ or : and then 0+ letters, digits, _, :, . or -
• (?:\s*=\s*(?:(?['"])(?<value>.*?)\k<q>|(?<value>(?:(?!/>)[^\s'"<>])+)))? - one or zero occurrence of (the value is thus optional):
  • \s*=\s* - a = enclosed with 0+ whitespaces
  • (?: - start of a non-capturing group:
    • (?[`'"]) - a delimiter, `, ' or "
    • (?<value>.*?) - Group "value" matching any 0+ chars other than line break chars as few as possible
    • \k<q> - backreference to Group "q", same value must match
  • | - or
    • (?<value>(?:(?!/>)[^\s`'"<>])+) - Group "value": a char other than whitespace, `, ', ", < and >, 1 or more occurrences, that does not start a /> char sequence
• ) - end of the non-capturing group.