Round-trip parsing of data structure format (YAML or whatnot) preserving comments, for writing configuration

Question

I have been using YAML as configuration file format in several applications, and all went well except one thing: when my program needs to write/modify a config variable in a YAML config file, it destroys formatting and comments by loading and dumping the entire file/structure.

(Well, there is another problem with YAML actually. Most users, many of them are not programmers, will be tripped over the details of YAML rules, like the significance of whitespace in some places. But this is not a major gripe.)

What I would prefer is a YAML loader/dumper which can do round-trip parsing (preserving all whitespaces & comments), or some other human-readable serialization format which has such parser. I'm even considering using Perl document and PPI, since PPI is a round-trip safe parser. Or perhaps PPI can be bent to deal with YAML or similar formats? I'd rather not use XML, I'd resort to INI+(JSON|YAML|... for key values) before that.

Any advice or pointers?

Answer 1

If you are using block structured YAML and Python is acceptable, you can use the Python package¹ ruamel.yaml which is a derivative of PyYAML and supports round trip preservation of comments:

import sys
import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

yaml = ruamel.yaml.YAML()

code = yaml.load(inp)
code['name']['given'] = 'Bob'

yaml.dump(code, sys.stdout)

with result:

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Note that the end-of-line comments are still aligned.

Instead of normal list and dict objects the code consists of wrapped versions² on which the comments attached.

¹ _{Install with pip install ruamel.yaml. Works on Python 2.6/2.7/3.3+. Disclaimer: I am the author of that package.}
² _{ordereddict is used in case of a mapping, to preserve ordering}

Answer 2

Yeah, you and everyone who thought wow, yaml sounds cool, simply put, it doesn't exist, yet

update: you probably want to use Config::General, its apache config format (xmlish)

No, PPI is not general purpose tool, if you want BNF-ness, you want to use Marpa

Of all INI/JSON/YAML/XML, XML probably has the best editor support for non-programmers (sounds crazy)