Remove all null values

Question

I am trying to remove null values from a json object using jq. I found this issue on their github and so now I'm trying to remove them with del. I have this:

'{ id: $customerId, name, phones: ([{ original: .phone }, 
 { original: .otherPhone}]), email} | del(. | nulls)'

This doesn't seem to do anything. However if I replace nulls with .phones it does remove the phone numbers.

Answer 1

The following illustrates how to remove all the null-valued keys from a JSON object:

jq -n '{"a":1, "b": null, "c": null} | with_entries( select( .value != null ) )'
{
  "a": 1
}

Alternatively, paths/0 can be used as follows:

. as $o | [paths[] | {(.) : ($o[.])} ] | add

By the way, del/1 can also be used to achieve the same result, e.g. using this filter:

reduce keys[] as $k (.; if .[$k] == null then del(.[$k]) else . end)

Or less obviously, but more succinctly:

del( .[ (keys - [paths[]])[] ] )

And for the record, here are two ways to use delpaths/1:

jq -n '{"a":1, "b": null, "c": null, "d":2} as $o
  | $o
  | delpaths( [  keys[] | select( $o[.] == null ) ] | map( [.]) )'


$ jq -n '{"a":1, "b": null, "c": null, "d":2}
  | [delpaths((keys - paths) | map([.])) ] | add'

In both these last two cases, the output is the same: { "a": 1, "d": 2 }

Answer 2

This answer by Michael Homer on https://unix.stackexchange.com has a super concinse solution which works since jq 1.6:

del(..|nulls)

It deletes all null-valued properties (and values) from your JSON. Simple and sweet :)

nulls is a builtin filter and can be replaced by custom selects:

del(..|select(. == "value to delete"))

To remove elements based on multiple conditions, e.g. remove all bools and all numbers:

del(..|booleans,numbers)

or, to only delete nodes not matching a condition:

del(..|select(. == "value to keep" | not))

(The last example is only illustrative – of course you could swap == for !=, but sometimes this is not possible. e.g. to keep all truthy values: del(..|select(.|not)))

Answer 3

All the other answers to date here are workarounds for old versions of jq, and it isn't clear how do do this simply in the latest released version. In JQ 1.6 or newer this will do the job to remove nulls recursively:

$ jq 'walk( if type == "object" then with_entries(select(.value != null)) else . end)' input.json

Sourced from this comment on the issue where adding the walk() function was discussed upstream.

Answer 4

[WARNING: the definition of walk/1 given in this response is problematic, not least for the reason given in the first comment; note also that jq 1.6 defines walk/1 differently.]

I am adding the new answer to emphasize the extended version of the script by @jeff-mercado. My version of the script assumes the empty values are as follows:

• null;
• [] - empty arrays;
• {} - empty objects.

Removing of empty arrays and objects was borrowed from here https://stackoverflow.com/a/26196653/3627676.

def walk(f):
    . as $in | 
    if type == "object" then
        reduce keys[] as $key
            ( {}; . + { ($key): ( $in[$key] | walk(f) ) } ) | f
    elif type == "array" then 
        select(length > 0) | map( walk(f) ) | f
    else 
        f
    end;

walk(
    if type == "object" then
        with_entries(select( .value != null and .value != {} and .value != [] ))
    elif type == "array" then
        map(select( . != null and . != {} and .!= [] ))
    else
        .
    end
)