How to recursively remove certain keys from a multi-dimensional(depth not known) python dictionary?

Question

I'm using kendoUI Grid in one of my projects. I retrieved a piece of data using their api and found that it added some "unwanted" data to my json/dictionary. After passing the json back to my Pyramid backend, I need to remove these keys. The problem is, the dictionary can be of whatever depth and I don't know the depth in advance.

Example:

product = {
    id: "PR_12"
    name: "Blue shirt",
    description: "Flowery shirt for boys above 2 years old",
    _event: {<some unwanted data here>},
    length: <some unwanted data>,
    items: [{_event: {<some rubbish data>}, length: <more rubbish>, price: 23.30, quantity: 34, color: "Red", size: "Large"}, {_event: {<some more rubbish data>}, length: <even more rubbish>, price: 34.50, quantity: 20, color: "Blue", size: "Large"} ....]
}

I want to remove two keys in particular: "_event" & "length". I tried writing a recursive function to remove the data but I can't seem to get it right. Can someone please help?

Here's what I have:

def remove_specific_key(the_dict, rubbish):
  for key in the_dict:
    if key == rubbish:
      the_dict.pop(key)
    else:
      # check for rubbish in sub dict
      if isinstance(the_dict[key], dict):
        remove_specific_key(the_dict[key], rubbish)

      # check for existence of rubbish in lists
      elif isinstance(the_dict[key], list):
        for item in the_dict[key]:
          if item == rubbish:
            the_dict[key].remove(item)
   return the_dict

Answer 1

If you allow remove_specific_key (renamed remove_keys) to accept any object as its first argument, then you can simplify the code:

def remove_keys(obj, rubbish):
    if isinstance(obj, dict):
        obj = {
            key: remove_keys(value, rubbish) 
            for key, value in obj.iteritems()
            if key not in rubbish}
    elif isinstance(obj, list):
        obj = [remove_keys(item, rubbish)
                  for item in obj
                  if item not in rubbish]
    return obj

Since you wish to remove more than one key, you might as well let rubbish be a set instead of one particular key. With the above code, you'd remove '_event' and 'length' keys with

product = remove_keys(product, set(['_event', 'length']))

---

Edit: remove_key uses dict comprehension, introduced in Python2.7. For older version of Python, the equivalent would be

    obj = dict((key, remove_keys(value, rubbish))
               for key, value in obj.iteritems()
               if key not in rubbish)

Answer 2

Modifying a dict as you iterate it bad, an unnecessary, since you know exactly what key you are looking for. Also, your list of dicts aren't being handled right:

def remove_specific_key(the_dict, rubbish):
    if rubbish in the_dict:
        del the_dict[rubbish]
    for key, value in the_dict.items():
        # check for rubbish in sub dict
        if isinstance(value, dict):
            remove_specific_key(value, rubbish)

        # check for existence of rubbish in lists
        elif isinstance(value, list):
            for item in value:
                if isinstance(item, dict):
                    remove_specific_key(item, rubbish)