Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

JQ - removing duplicates using unique_by

Tags:

json

unique

jq

I am trying to remove the duplicates from the following json by id Here is the json:

{
  "Result": [
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 4740302
    },
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 272320
    },
    {
      "name": "validation-of-art",
      "id": "13",
      "status": "passed",
      "duration": 272320
    }
  ]
}

Here is what i have tried with:

jq -r 'unique_by(.Result.name)'

and also with jq 'unique_by(.Result[].name)'

I am getting an error - Cannot index array with string "Result" Any help would be appreciated.

like image 857
Saurabh Avatar asked Sep 25 '17 04:09

Saurabh


1 Answers

Here is an example which eliminates all but one of the .Result objects using unique_by(.name)

$ jq -M '.Result |= unique_by(.name)' data.json
{
  "Result": [
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 4740302
    }
  ]
}

If this isn't quite what you want you can generalize this easily. E.g. to keep one object for each unique {name,id} you could use

$ jq -M '.Result |= unique_by({name, id})' data.json
{
  "Result": [
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 4740302
    },
    {
      "name": "validation-of-art",
      "id": "13",
      "status": "passed",
      "duration": 272320
    }
  ]
}
like image 120
jq170727 Avatar answered Oct 30 '22 16:10

jq170727