How to scrub a dynamic python dictionary of specific keys?

Question

This one is just obtuse enough that no typical answer is bubbling to the top.

The nature of the tool I'm working on dictates we use MongoDB to store 'settings' for about 25 different tools. Each tool has it's own settings schema, so each document is different, but they're all stored in the same collection and edited on the same edit page which is drawn by json schema.

Not knowing the schema of the dictionary, I'm struggling to figure out how to iterate and sanitize the data, specifically removing passwords.

Given the following dictionary, and knowing that other dicts may have different schemas, how could I traverse every single item in the dict and create a copy, identical except with any key == "password" removed?

So:

{
  "_enabled": true,
  "instances": [
    {
      "isdefault": true,
      "name": "dev",
      "password": "abc123",
      "url": "http://dev.example.com",
      "user": "buffy"
    },
    {
      "isdefault": false,
      "name": "prod",
      "password": "xxxxx",
      "url": "http://prod.example.com",
      "user": "spike"
    },
    {
      "isdefault": false,
      "name": "qa",
      "password": "dasddf",
      "url": "http://prod.example.com",
      "user": "willow"
    }
  ],
  "label": "MyServers"
}

Should result in:

{
  "_enabled": true,
  "instances": [
    {
      "isdefault": true,
      "name": "dev",
      "url": "http://dev.example.com",
      "user": "buffy"
    },
    {
      "isdefault": false,
      "name": "prod",
      "url": "http://prod.example.com",
      "user": "spike"
    },
    {
      "isdefault": false,
      "name": "qa",
      "url": "http://prod.example.com",
      "user": "willow"
    }
  ],
  "label": "MyServers"
}

Answer 1

deepcopy the dict first, then catch all the dictionaries and remove the password keys:

from copy import deepcopy

def remove_pass(v):
    if isinstance(v, dict):
        if "password" in v:
            del v["password"]
        for ele in v.values():
            remove_pass(ele)
    elif isinstance(v, Iterable) and not isinstance(v, basestring):
        for ele in v:
            remove_pass(ele)


from pprint import pprint as pp
d = deepcopy(d)
for v in d.values():
   remove_pass(v)

Input:

{'_enabled': 'true',
 'foo': {'isdefault': 'false',
         'name': 'qa',
         'nested': {'password': 'nested'},
         'password': 'dasddf',
         'url': 'http://prod.example.com',
         'user': 'willow'},
 'instances': [{'isdefault': 'true',
                'name': 'dev',
                'password': 'abc123',
                'url': 'http://dev.example.com',
                'user': 'buffy'},
               {'isdefault': 'false',
                'name': 'prod',
                nested': {'more_nesting': {'even_more_nesting': ({'password': 'foobar'},
                                       {'password': 'foob'}),
                                        'password': 'bar'},
                'password': 'xxxxx',
                'url': 'http://prod.example.com',
                'user': 'spike'},
               {'isdefault': 'false',
                'name': 'qa',
                'password': 'dasddf',
                'url': 'http://prod.example.com',
                'user': 'willow'}],
 'label': 'MyServers'}

Output:

{'_enabled': 'true',
 'foo': {'isdefault': 'false',
         'name': 'qa',
         'nested': {},
         'url': 'http://prod.example.com',
         'user': 'willow'},
 'instances': [{'isdefault': 'true',
                'name': 'dev',
                'url': 'http://dev.example.com',
                'user': 'buffy'},
               {'isdefault': 'false',
                'name': 'prod',
                'nested': {'more_nesting': {'even_more_nesting': ({}, {})}},
                'url': 'http://prod.example.com',
                'user': 'spike'},
               {'isdefault': 'false',
                'name': 'qa',
                'url': 'http://prod.example.com',
                'user': 'willow'}],
 'label': 'MyServers'}