Is there YAML syntax for sharing part of a list or map?

Question

So, I know I can do something like this:

sitelist: &sites
  - www.foo.com
  - www.bar.com

anotherlist: *sites

And have sitelist and anotherlist both contain www.foo.com and www.bar.com. However, what I really want is for anotherlist to also contain www.baz.com, without having to repeat www.foo.com and www.baz.com.

Doing this gives me a syntax error in the YAML parser:

sitelist: &sites
  - www.foo.com
  - www.bar.com

anotherlist: *sites
  - www.baz.com

Just using anchors and aliases it doesn't seem possible to do what I want without adding another level of substructure, such as:

sitelist: &sites
  - www.foo.com
  - www.bar.com

anotherlist:
  - *sites
  - www.baz.com

Which means the consumer of this YAML file has to be aware of it.

Is there a pure YAML way of doing something like this? Or will I have to use some post-YAML processing, such as implementing variable substitution or auto-lifting of certain kinds of substructure? I'm already doing that kind of post-processing to handle a couple of other use-cases, so I'm not totally averse to it. But my YAML files are going to be written by humans, not machine generated, so I would like to minimise the number of rules that need to be memorised by my users on top of standard YAML syntax.

I'd also like to be able to do the analogous thing with maps:

namedsites: &sites
  Foo: www.foo.com
  Bar: www.bar.com

moresites: *sites
  Baz: www.baz.com

I've had a search through the YAML spec, and couldn't find anything, so I suspect the answer is just "no you can't do this". But if anyone has any ideas that would be great.

---

EDIT: Since there have been no answers, I'm presuming that no one has spotted anything I haven't in the YAML spec and that this can't be done at the YAML layer. So I'm opening up the question to idea for post-processing the YAML to help with this, in case anyone finds this question in future.

Answer 1

The merge key type is probably what you want. It uses a special << mapping key to indicate merges, allowing an alias to a mapping (or a sequence of such aliases) to be used as an initializer to merge into a single mapping. Additionally, you can still explicitly override values, or add more that weren't present in the merge list.

It's important to note that it works with mappings, not sequences as your first example. This makes sense when you think about it, and your example looks like it probably doesn't need to be sequential anyway. Simply changing your sequence values to mapping keys should do the trick, as in the following (untested) example:

sitelist: &sites
  ? www.foo.com  # "www.foo.com" is the key, the value is null
  ? www.bar.com

anotherlist:
  << : *sites    # merge *sites into this mapping
  ? www.baz.com  # add extra stuff

Some things to notice. Firstly, since << is a key, it can only be specified once per node. Secondly, when using a sequence as the value, the order is significant. This doesn't matter in the example here, since there aren't associated values, but it's worth being aware.

Answer 2

As the previous answers have pointed out, there is no built-in support for extending lists in YAML. I am offering yet another way to implement it yourself. Consider this:

defaults: &defaults
  sites:
    - www.foo.com
    - www.bar.com
     
setup1:
  <<: *defaults
  sites+:
    - www.baz.com

This will be processed into:

defaults:
  sites:
    - www.foo.com
    - www.bar.com

setup1:
  sites:
    - www.foo.com
    - www.bar.com
    - www.baz.com

The idea is to merge the contents of a key ending with a '+' to the corresponding key without a '+'. I implemented this in Python and published here.

Answer 3

(Answering my own question in case the solution I'm using is useful for anyone who searches for this in future)

With no pure-YAML way to do this, I'm going to implement this as a "syntax transformation" sitting between the YAML parser and the code that actually uses the configuration file. So my core application doesn't have to worry at all about any human-friendly redundancy-avoidance measures, and can just act directly on the resulting structures.

The structure I'm going to use looks like this:

foo:
  MERGE:
    - - a
      - b
      - c
    - - 1
      - 2
      - 3

Which would be transformed to the equivalent of:

foo:
  - a
  - b
  - c
  - 1
  - 2
  - 3

Or, with maps:

foo:
  MERGE:
    - fork: a
      spoon: b
      knife: c
    - cup: 1
      mug: 2
      glass: 3

Would be transformed to:

foo:
  fork: a
  spoon: b
  knife: c
  cup: 1
  mug: 2
  glass: 3

More formally, after calling the YAML parser to get native objects from a config file, but before passing the objects to the rest of the application, my application will walk the object graph looking for mappings containing the single key MERGE. The value associated with MERGE must be either a list of lists, or a list of maps; any other substructure is an error.

In the list-of-lists case, the entire map containing MERGE will be replaced by the child lists concatenated together in the order they appeared.

In the list-of-maps case, the entire map containing MERGE will be replaced by a single map containing all of the key/value pairs in the child maps. Where there is overlap in the keys, the value from the child map occurring last in the MERGE list will be used.

The examples given above are not that useful, since you could have just written the structure you wanted directly. It's more likely to appear as:

foo:
  MERGE:
    - *salt
    - *pepper

Allowing you to create a list or map containing everything in nodes salt and pepper being used elsewhere.

(I keep giving that foo: outer map to show that MERGE must be the only key in its mapping, which means that MERGE cannot appear as a top-level name unless there are no other top level names)