Processing shell-style comments in JSON

Question

I'm dealing with systems which manipulate "relaxed" JSON data which includes shell-style # line comments:

[
  {
    # Batman
    "first-name": "Bruce",
    "last-name": "Wayne"
  },
  {
    # Superman
    "first-name": "Clark",
    "last-name": "Kent"
  }
]

The part of the system I'm working on uses json-lib - which I'm surprised to discover is tolerant of the shell-style comments - to parse the JSON input.

I need to extract some additional annotation from those comments, but json-lib seems to just discard them without providing an API for reading them:

JSONObject map = (JSONObject)JSONSerializer.toJSON("{\n"+
                                                   "    # Batman\n" + // note the shell-style # comment
                                                   "    \"first-name\": \"Bruce\",\n" +
                                                   "    \"last-name\": \"Wayne\"\n" +
                                                   "}");
System.out.println(map.toString());
/* <<'OUTPUT'
 * {"first-name":"Bruce","last-name":"Wayne"}
 * OUTPUT
 * note the absence of the shell-style comment
 */

This makes sense since comments aren't part of the JSON spec and I'm lucky json-lib doesn't just choke when parsing them in the first place.

Of note:

• other systems consume this same JSON and the annotations need to be transparent to them, so the JSON structure can't be modified by adding properties for the comments instead.
• not all the components and objects in my system have access to the raw JSON source: one component reads the file and parses it using JSONlib and passes de-serialized maps etc around.

How can I read and parse these comments while processing the JSON input? Is there a library which will allow me to read them and relate them to their position in the JSON - can I easily connect the Batman comment to the "Bruce Wayne" entry?

I'm currently using json-lib, but I'm open to investigating other JSON libraries and equally open to using other languages which extend JSON, such as YAML - but I'm not sure those tools will allow me to read and process the comments in my input.

Answer 1

EDIT (May 27, 2021):

What I chose to do is write a custom JSON parser for this nonstandard version of JSON. It supports the shell comments given in your question, but only before the first key of a JSON object:

• https://github.com/peteroupc/CBOR/blob/master/CBORTest/JSONWithComments.cs
• https://github.com/peteroupc/CBOR/blob/master/CBORTest/JSONPointer.cs

(The preceding is in C#; a Java version is expected to be available as well. It relies on my Concise Binary Object Representation library, called PeterO.Cbor in NuGet or com.upokecenter/cbor in the Central Repository.)

Indeed, one of your requirements is that "the JSON structure can't be modified by adding properties for the comments instead." That means the comments must be associated to the JSON objects in some other way. Fortunately, a specification called JSON Pointer was recently published as RFC 6901. JSON Pointer is a string that refers to a JSON object within another JSON object. This is why the parser includes a way to get the comment and its associated JSON pointer via a method called JSONWithComments.FromJSONStringWithPointers. JSONPointer.cs is my own implementation of the JSON Pointer specification.

Example of use:

      dict=new Dictionary<string, string>();
      str="{\"f\":[\n {\n # B\t \tA C\n # Dm\n\"a\":1,\n\"b\":2\n},{\n #" +
"\u0020Sm\n\"a\":3,\n\"b\":4\n}\n]}";
       obj = JSONWithComments.FromJSONString(str);
      Console.WriteLine(obj);
       obj = JSONWithComments.FromJSONStringWithPointers(str, dict);
       // Get the comment and its associated JSON pointer
       foreach(string key in dict.Keys) {
         Console.WriteLine(key);
         Console.WriteLine(dict[key]);
         // Get the pointed-to object
         Console.WriteLine(JSONPointer.GetObject(obj,dict[key]));
       }
       // Output the object
      Console.WriteLine(obj);