How to automatically re-indent a YAML file?

Question

Let's consider this example:

---
foo:
  alice: female
  bob:   male
bar:
  - node: 42
    name: none
  - node: 43
    name: none

If I decide to reindent everything and start from this:

---
foo:
alice: female
bob:   male
bar:
- node: 42
name: none
- node: 43
name: none

I don't have enough information to do it. Actually vim's attempt is this:

---
foo:
    alice: female
bob:   male
bar:
    - node: 42
name: none
- node: 43
name: none

Ansible cannot make it as well:

---
foo:

alice: female
bob:   male
bar:

- node: 42
  name: none
  - node: 43
    name: none

I think a possible workaround would be to add a blank line when indentation is increased:

---
foo:

  alice: female
  bob:   male
bar:

  - node: 42
    name: none
  - node: 43
    name: none

In my opinion, the main problem with YAML is that people are still fighting with tabs, spaces and indent size. For large YAML files that are edited by several people, the resulting file cannot be properly parsed anymore. The two solutions I see are:

1. Use only tabs and force tabs identation as this is the case with Makefiles
2. The above solution

Answer 1

What you seem to want to do is making sure that your YAML files are uniformly indented (e.g. before being checked into a revision control system). Your idea of dedenting and then re-indenting will not work as you lose information if you flatten your structure. This:

foo:
  alice: female
  bob:   male

consists of two mappings: a mapping with one key and a value that is mapping of two keys to two values.

This:

foo:
alice: female
bob:   male

is one mapping with three keys, and key foo has as value the null scalar (also writable, apart from the empty string, as ~, NULL, null in YAML files).

Most YAML parsers will lose information when reading in a file into internal data:

• comments are dropped
• key ordering is not preserved for mappings
• extra spaces around scalars are not preserved

The ruamel.yaml Python package (of which I am the author) is an enhancemed parser which to allows round-tripping a YAML file to data and back to YAML to preserve more of the original information. It will preserve comments and key ordering, but it drops e.g. extra spacing around single line scalars.

This round-tripping normally stabilizes on a second round-trip and so this can be used to reindent a YAML file. The yaml utility included in the package ruamel.yaml.cmd, can be used for that without the need to program things yourself:

yaml round-trip your_file.yml --verbose

(round-trip can be shortened to rt) will check whether and how the file would change. It shows a unified diff if it does change. Based on that you can decide to save the file if it stabilizes:

yaml round-trip your_file.yml --save

the output for example.yml:

---
foo:
  alice: female  # verified
  bob:   male
bar:
- node:   42
  name: none
-     node: 43
      name: none

would be:

example.yml:
     stabilzes on second round trip, ok without comments
--- example.yml
+++ round trip YAML
@@ -1,9 +1,9 @@
 ---
 foo:
   alice: female  # verified
-  bob:   male
+  bob: male
 bar:
-- node:   42
+- node: 42
   name: none
--     node: 43
-      name: none
+- node: 43
+  name: none

and when saved look like:

---
foo:
  alice: female  # verified
  bob: male
bar:
- node: 42
  name: none
- node: 43
  name: none

The indentation level is by default 2, but can be set with an option to yaml.

Answer 2

I just found another solution, it requires a couple of regular expression replacements.

First choose a single character not included in the yaml. I used "°"

Here I wanted to reindent 4 spaces to 2 spaces:

Search regex "^(( )*)( )" (4 spaces in braces) and replace with "$1°" until nothing left to replace.

Replace once all "°" with "" (2spaces).. Voila!

What happens is that the rightmost indent is sub-sequentially replaced by a placeholder. To avoid incorrect matching of already replaced stuff a placeholder not matching to the search is required.

PS: I know it's not fully automated but most editors have regex replace so it requires no extra installation.