compare 2 json arrays and return the difference

Question

We have a custom CD Pipeline Tool, which unfortunately does not version the deployment parameters. So I put these in a Bitbucket Repo as a json file and validate them against a REST API of this CD Tool.

So I have 2 json arrays, which are structurally the same, but may contain different objects or values in these objects. I want to compare them to see if they are different and what is different.

So far, I used the solution from here: Using jq or alternative command line tools to diff JSON files

So I have put this in my code:

jq --argjson a "${bb_cfg}" --argjson b "${cd_tool_cfg}" -n 'def post_recurse(f): def r: (f | select(. != null) | r), .; r; def post_recurse: post_recurse(.[]?); ($a | (post_recurse | arrays) |= sort) as $a | ($b | (post_recurse | arrays) |= sort) as $b | $a == $b'

now I get a true if they are identical or false if 2 jsons have differences, but I do not know what is different.

I tried to do this with this if I get false back:

diff --suppress-common-lines -y <(jq . -S <<< "${bb_cfg}") <(jq . -S <<< "${cd_tool_cfg}")

Input $bb_cfg:

[{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "true",
    "tags": []
},
{
    "key": "BB_CFG_REPO_NAME",
    "value": "cd-tool-cfg",
    "tags": []
}]

Input $cd_tool_cfg

[{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "false",
    "tags": []
},
{
    "key": "BB_CFG_REPO_NAME",
    "value": "cd-tool-cfg",
    "tags": []
}]

which works partly, because if only the value is different, the output is like this:

    "value": "true"                       |     "value": "false"

so I do not get the whole json object here to quickly find out what parameter is different.

What I eventually want is to get something like this:

{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "true",
    "tags": []
}
{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "false",
    "tags": []
}

where I can store this in a variable in my bash script and transform this in an output I can use.

Answer 1

You could use jq's -c or --compact-output option:

diff <(jq -c .[] <<<"$bb_cfg") <(jq -c .[] <<<"$cd_tool_cfg")
1c1
< {"key":"IGNORE_VALIDATION_ERROR","value":"true","tags":[]}
---
> {"key":"IGNORE_VALIDATION_ERROR","value":"false","tags":[]}

The -c option will simply output a json with each array member on a separate line.

The following command will give you something like you requested:

diff --old-line-format="%L" --unchanged-line-format="" --new-line-format="%L" <(jq -c .[] <<<"$bb_cfg") <(jq -c .[] <<<"$cd_tool_cfg") | jq

will output:

{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "true",
    "tags": []
}
{
    "key": "IGNORE_VALIDATION_ERROR",
    "value": "false",
    "tags": []
}