Python JSON - finding elements on a dynamic path

Question

I have to update a nested JSON object.
If I knew the specifics of which items were to be updated I could do :

json_object['basket']['items']['apple'] = 'new value'

However, my list of elements to target is dynamic.

> basket.items.apple = 'green'
> name = 'my shopping'
> basket.cost = '15.43'

I could do this by looping through elements.
Find 'basket' > then find 'items > then find 'apple' > set value
Find 'name' > set value

However, was hoping that there was a way to just reference directly/dynamicaly.
i.e. from a string 'basket.cost', build the expression :

json_object['basket']['cost']

P.s. it has to cope with lists of dictionaries too !

Any guidance appreciated :)

Answer 1

Once you have the string "basket.cost", you can split it on "." and it's pretty easy to drill down into json_object['basket']['cost'] using a loop. Functionally, there is no difference between doing this and doing it "directly": you are still getting the 'basket' key first, and then getting the 'cost' key from the value of json_object['basket'].

def get_element(d, path):
    # This function can take the string "basket.cost", or the list ["basket", "cost"]
    if isinstance(path, str):
        path = path.split(".")

    for p in path:
        d = d[p]

    return d

def set_element(d, path, value):
    path = path.split(".")
    dict_to_set = get_element(d, path[:-1])
    key_to_set = path[-1]
    dict_to_set[key_to_set] = value

set_element(json_object, "basket.items.apple", 100)

Now, this assumes all elements of your path already exist, so let's say you create a dictionary that looks like so:

json_object = {"basket": {"items": dict()}}
set_element(json_object, "basket.items.apple", 100)
set_element(json_object, "basket.cost", 10)
print(json_object) 
# Output: {'basket': {'items': {'apple': 100}, 'cost': 10}}

print(get_element(json_object, "basket.cost")) 
# Output: 10

If you try to access an element that doesn't already exist, you get a KeyError:

get_element(json_object, "basket.date")
# KeyError: 'date'

This also happens if you try to set a value in an element that doesn't exist:

set_element(json_object, "basket.date.day", 1)
# KeyError: 'date'

---

If we want to allow your function to create the dictionaries when they don't exist, we can modify the get_element function to account for this situation and add the key:

def get_element(d, path, create_missing=False):
    # This function can take the string "basket.cost", or an iterable containing the elements "basket" and "cost"
    if isinstance(path, str):
        path = path.split(".")

    for p in path:
        if create_missing and p not in d:
            d[p] = dict()

        d = d[p]

    return d

def set_element(d, path, value, create_missing=True):
    path = path.split(".")
    dict_to_set = get_element(d, path[:-1], create_missing)
    key_to_set = path[-1]
    dict_to_set[key_to_set] = value

set_element(json_object, "basket.date.day", 1)
print(json_object)
# Output: {'basket': {'items': {'apple': 100}, 'cost': 10, 'date': {'day': 1}}}

Answer 2

If using third party package is an option, you can try python-box. It comes with lots of options and utilities to load from json, yaml files. The implementation is optimized for speed using Cython.

from box import Box

test_data = {
    "basket": {
        "products": [
            {"name": "apple", "colour": "green"}
        ],
    }
}

a = Box(test_data)
a.basket.cost = 12.3
a.basket.products[0].colour = "pink"
a.basket.products.append({"name": "pineapple", "taste": "sweet"})
print(a.basket.products[1].taste)