How can I tell if a jq filter successfully pulls data from a JSON data structure?

Question

I want to know if a given filter succeeds in pulling data from a JSON data structure. For example:

###### For the user steve...

% Name=steve
% jq -j --arg Name "$Name" '.[]|select(.user == $Name)|.value' <<<'
[
   {"user":"steve", "value":false},
   {"user":"tom", "value":true},
   {"user":"pat", "value":null},
   {"user":"jane", "value":""}
]'
false
% echo $?
0

Note: successful results can include boolean values, null, and even the empty string.

###### Now for user not in the JSON data...

% Name=mary
% jq -j --arg Name "$Name" '.[]|select(.user == $Name)|.value' <<<'
[
   {"user":"steve", "value":false},
   {"user":"tom", "value":true},
   {"user":"pat", "value":null},
   {"user":"jane", "value":""}
]'
% echo $?
0

If the filter does not pull data from the JSON data structure, I need to know this. I would prefer the filter to return a non-zero return code.

How would I go about determining if a selector successfully pulls data from a JSON data structure vs. fails to pull data?

Important: The above filter is just an example, the solution needs to work for any jq filter.

Note: the evaluation environment is Bash 4.2+.

Answer 1

You can use the -e / --exit-status flag from the jq Manual, which says

Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.

I can demonstrate the usage with a basic filter as below, as your given example is not working for me.

For a successful query,

dudeOnMac:~$ jq -e '.foo?' <<< '{"foo": 42, "bar": "less interesting data"}'
42
dudeOnMac:~$ echo $?
0

For an invalid query, done with a non-existent entity zoo,

dudeOnMac:~$ jq -e '.zoo?' <<< '{"foo": 42, "bar": "less interesting data"}'
null
dudeOnMac:~$ echo $?
1

For an error scenario, returning code 2 which I created by double-quoting the jq input stream.

dudeOnMac:~$ jq -e '.zoo?' <<< "{"foo": 42, "bar": "less interesting data"}"
jq: error: Could not open file interesting: No such file or directory
jq: error: Could not open file data}: No such file or directory
dudeOnMac:~$ echo $?
2

Answer 2

I've added an updated solution below

The fundamental problem here is that when try to retrieve a value from an object using the .key or .[key] syntax, jq — by definition — can't distinguish a missing key from a key with a value of null.

You can instead define your own lookup function:

def lookup(k):if has(k) then .[k] else error("invalid key") end;

Then use it like so:

$ jq 'lookup("a")' <<<'{}' ; echo $?
jq: error (at <stdin>:1): invalid key
5

$ jq 'lookup("a")' <<<'{"a":null}' ; echo $?
null
0

If you then use lookup consistently instead of the builtin method, I think that will give you the behaviour you want.

---

Here's another way to go about it, with less bash and more jq.

#!/bin/bash

lib='def value(f):((f|tojson)//error("no such value"))|fromjson;'

users=( steve tom pat jane mary )

Select () {
  local name=$1 filter=$2 input=$3
  local -i status=0
  result=$( jq --arg name "$name" "${lib}value(${filter})" <<<$input  2>/dev/null )
  status=$? 
  (( status )) && result="***ERROR***"
  printf '%s\t%d %s\n' "$name" $status "$result"
}

filter='.[]|select(.user == $name)|.value'

input='[{"user":"steve","value":false},
        {"user":"tom","value":true},
        {"user":"pat","value":null},
        {"user":"jane","value":""}]'

for name in "${users[@]}"
do
  Select "$name" "$filter" "$input"
done

This produces the output:

steve   0 false
tom     0 true
pat     0 null
jane    0 ""
mary    5 ***ERROR***

This takes advantage of the fact the absence of input to a filter acts like empty, and empty will trigger the alternative of //, but a string — like "null" or "false" — will not.

It should be noted that value/1 will not work for filters that are simple key/index lookups on objects/arrays, but neither will your solution. I'm reasonably sure that to cover all the cases, you'd need something like this (or yours) and something like get or lookup.