How to filter JSON data using Python?

Question

How to convert JSON data from input.json to output.json using Python? In general, what data structures are used for filtering JSON data?

File: input.json

[
{
    "id":1,
    "a":22,
    "b":11
},
{
    "id":1,
    "e":44,
    "c":77,
    "f":55,
    "d":66
},
{
    "id":3,
    "b":11,
    "a":22
},
{
    "id":3,
    "d":44,
    "c":88
}
]

File: output.json

[
{
    "id":1,
    "a":22,
    "b":11,
    "e":44,
    "c":77,
    "f":55,
    "d":66
},
{
    "id":3,
    "b":11,
    "a":22,
    "d":44,
    "c":88
}
]

Any pointers would be appreciated!

Answer 1

The idea is to:

• use json.load() to load the JSON content from file to a Python list
• regroup the data by the id, using collections.defaultdict and .update() method
• use json.dump() to dump the result into the JSON file

Implementation:

import json
from collections import defaultdict

# read JSON data
with open("input.json") as input_file:
    old_data = json.load(input_file)

# regroup data
d = defaultdict(dict)
for item in old_data:
    d[item["id"]].update(item)

# write JSON data
with open("output.json", "w") as output_file:
    json.dump(list(d.values()), output_file, indent=4)

Now the output.json would contain:

[
    {
        "d": 66,
        "e": 44,
        "a": 22,
        "b": 11,
        "c": 77,
        "id": 1,
        "f": 55
    },
    {
        "b": 11,
        "id": 3,
        "d": 44,
        "c": 88,
        "a": 22
    }
]