jq: recursively merge objects and concatenate arrays

Question

I have two json files, orig.json and patch.json, which have similar formats.

orig.json:

{
        "a": {
                "a1": "original a1",
                "a2": "original a2",
                "list": ["baz", "bar"]
        },
        "b": "original value B"
}

patch.json:

{
    "a": {
            "a1": "patch a1",
            "list": ["foo"]
    },
    "c": "original c"
}

Currently I am using jq to merge them recursively. However, jq's default behavior for lists is just reassignment. Example output from jq using $ jq -s '.[0] * .[1]' orig.json patch.json:

{
  "a": {
    "a1": "patch a1",
    "a2": "original a2",
    "list": [
      "foo"
    ]
  },
  "b": "original value B",
  "c": "original c"
}

Note that a.list is now equal to patch.json's a.list. I want The new a.list to be orig.json's list and patch.json's lists merged. In other words, I want a.list to equal ["baz", "bar", "foo"].

Is there a way I can easily do this with jq, perhaps by overriding the default merge strategy for arrays?

Answer 1

Here is a generic function that recursively combines two composite JSON entities by concatenating arrays at the same position:

# Recursively meld a and b,
# concatenating arrays and
# favoring b when there is a conflict 
def meld(a; b):
  a as $a | b as $b
  | if ($a|type) == "object" and ($b|type) == "object"
    then reduce ([$a,$b]|add|keys_unsorted[]) as $k ({}; 
      .[$k] = meld( $a[$k]; $b[$k]) )
    elif ($a|type) == "array" and ($b|type) == "array"
    then $a+$b
    elif $b == null then $a
    else $b
    end;

Output of meld($orig; $patch)

With $orig set to the contents of orig.json and $patch set to the contents of patch.json:

{
  "a": {
    "a1": "patch a1",
    "a2": "original a2",
    "list": [
      "baz",
      "bar",
      "foo"
    ]
  },
  "b": "original value B",
  "c": "original c"
}

Answer 2

$ jq -s 'def deepmerge(a;b):
  reduce b[] as $item (a;
    reduce ($item | keys_unsorted[]) as $key (.;
      $item[$key] as $val | ($val | type) as $type | .[$key] = if ($type == "object") then
        deepmerge({}; [if .[$key] == null then {} else .[$key] end, $val])
      elif ($type == "array") then
        (.[$key] + $val | unique)
      else
        $val
      end)
    );
  deepmerge({}; .)' file1.json file2.json > merged.json