Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flatten nested JSON array with JQ

Tags:

json

jq

I have a JSON in the following format:

{
  "@version": "2.7.0",
  "site": {
    "@name": "http://api:9999",
    "@ssl": "false",
    "alerts": [
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "instances": [
          {
            "uri": "http://api:9999",
            "method": "POST",
            "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg"
          },
          {
            "uri": "http://api:9999",
            "method": "POST",
            "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ"
          }
        ],
        "count": "37"
      }
    ]
  }
}

I want to flatten the inner array - .site.alerts.instances to get the following JSON:

{
    "@name": "http://api:9999",
    "@ssl": "false",
    "alerts": [
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "uri": "http://api:9999",
        "method": "POST",
        "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg",
        "count": "37"
      },
      {
        "pluginid": "10094",
        "desc": "<p>Base64 encoded data was disclosed by the application/web server<\/p>",
        "uri": "http://api:9999",
        "method": "POST",
        "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ",
        "count": "37"
      }
    ]
  }

I was able to flatten the inner JSON array by using the following JQ pattern:

.site.alerts[] as $in | $in.instances[] as $h |  $in | del(.instances) as $in2 |  $h * $in2 

This gave me a very close result:

{
  "uri": "http://api:9999",
  "method": "POST",
  "evidence": "DxyPP_YQ6qdWA_Kw_ZLgYilIkXCz93Xs1CeJPvg",
  "pluginid": "10094",
  "desc": "<p>Base64 encoded data was disclosed by the application/web server</p>",
  "count": "37"
}
{
  "uri": "http://api:9999",
  "method": "POST",
  "evidence": "eyJuYmYiOjE121lMWF1siSG9tZUFwcCJdfQ",
  "pluginid": "10094",
  "desc": "<p>Base64 encoded data was disclosed by the application/web server</p>",
  "count": "37"
}

But not a perfect result. The objects are not in an array, and fields from the parent object, which are not part of the array (e.g. .site.@name) are not included.

Can you help me improve the JQ pattern I created?

Thanks in advance!

like image 937
Omer Levi Hevroni Avatar asked Jun 23 '26 17:06

Omer Levi Hevroni


1 Answers

That's a really good effort made. Your idea is right, you've made sure the .instances[] array is flattened, just use that logic to re-construct the JSON as you need as

jq '{ "@name" : .site."@name", 
      "@ssl"  : .site."@ssl", 
      "alerts": [.site.alerts[] as $in | $in.instances[] as $h | $in | del(.instances) as $in2 | $h * $in2 ]}' json

jqplay.org - URL

like image 174
Inian Avatar answered Jun 25 '26 10:06

Inian



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!