Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Use JQ to select specific, arbitrarily nested objects from JSON

I'm looking for efficient means to search through an large JSON object for "sub-objects" that match a filter (via select(), I imagine). However, the top-level JSON is an object with arbitrary nesting contained within, including more simple values, objects and arrays of objects. For example:

{
  "name": "foo",
  "class": "system",
  "description": "top-level-thing",
  "configuration": {
    "status": "normal",
    "uuid": "id"
  },
  "children": [
    {
      "id": "c1",
      "class": "c1",
      "children": [
        {
          "id": "c1.1",
          "class": "c1.1"
        },
        {
          "id": "c1.1",
          "class": "FINDME"
        }
      ]
    },
    {
      "id": "c2",
      "class": "FINDME"
    }
  ],
  "thing": {
    "id": "c3",
    "class": "FINDME"
  }
}    

I have a solution which does part of what I want (and is understandable):

jq -r '.. | arrays | .[] | select(.class=="FINDME"?) | .id'

which returns:

c2
c1.1

... however, it misses c3, plus it changes the order of items output. Additionally I'm expecting this to operate on potentially very large JSON structures, I would like to make sure I find an efficient solution. Bonus points for something that remains readable by jq neophytes (myself included).

FWIW, references I was using to help me on the way, in case they help others:

  • Select objects based on value of variable in object using jq
  • How to use jq to find all paths to a certain key
  • Recursive search values by key
like image 801
crimson-egret Avatar asked Dec 19 '17 16:12

crimson-egret


1 Answers

For small to modest-sized JSON input, you're on the right track with .. but it seems you want to select objects, like so:

.. | objects | select(.class=="FINDME"?) | .id

For JSON documents that are very large, this might require too much memory, so it may be worth knowing about jq's streaming parser. Unfortunately it's much more difficult to use, so I'd suggest trying the above, and if you're interested, look in the usual places for documentation about the --stream option.

like image 76
peak Avatar answered Nov 14 '22 12:11

peak