Menu
Does anyone know how to use jq to find the duplicate(s) in a JSON array?
For example:
Input:
[{"foo": 1, "bar": 2}, {"foo": 1, "bar": 2}, {"foo": 4, "bar": 5}]
Output:
[{"foo": 1, "bar": 2}]
865
Duplicate elements can be found using two loops. The outer loop will iterate through the array from 0 to length of the array. The outer loop will select an element. The inner loop will be used to compare the selected element with the rest of the elements of the array.
The short answer: Yes but is not recommended. The long answer: It depends on what you call valid... [ECMA-404][1] "The JSON Data Interchange Syntax" doesn't say anything about duplicated names (keys).
We can use the JSON. stringify method to convert a plain JavaScript object into a string. This lets us check all the properties at once instead of hard coding the check as we did in the previous example. to remove the duplicate item by using JSON.
How to remove the duplicates? You will need to convert the JSON to Java Objects and then perform the duplicate removal operation. Added code snippet for each of the steps. Hope this helps!
One of many possible solutions in jq:
group_by(.) | map(select(length>1) | .[0])
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
