Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fastest way to convert a dict's keys & values from `bytes` to `str` in python3

Here's my attempt:

def convert(data):
    if isinstance(data, bytes):
        return data.decode('ascii')
    elif isinstance(data, dict):
        return dict(map(convert, data.items()))
    elif isinstance(data, tuple):
        return map(convert, data)
    else:
        return data

Can this be better generalized and or improve legibility?

like image 534
Guy Gangemi Avatar asked Oct 15 '15 00:10

Guy Gangemi


People also ask

How to find keys in a dictionary Python?

The get() method is a dictionary method that returns the value of the associated key. If the key is not present it returns either a default value (if passed) or it returns None. Using this method we can pass a key and check if a key exists in the python dictionary.

How to Get keys from list in Python?

You can get all the keys in the dictionary as a Python List. dict. keys() returns an iterable of type dict_keys() . You can convert this into a list using list() .


3 Answers

Don't know about optimisation for speed but I'm not a big fan of the if/return/else paradigm since it clogs up the code with unnecessary stuff and causes indentation staircases for languages without elif (not such a problem here).

Optimising for readability (which is usually my first preference), I'd turn all those elif lines into if and ditch the else entirely, reformatting to make it more compact:

def convert(data):
    if isinstance(data, bytes):  return data.decode('ascii')
    if isinstance(data, dict):   return dict(map(convert, data.items()))
    if isinstance(data, tuple):  return map(convert, data)
    return data
like image 154
paxdiablo Avatar answered Sep 18 '22 09:09

paxdiablo


The easiest way would be to use dictionary comprehension as follows:

new_data = { key.decode(): val.decode() for key, val in data.items() }

Example:

>>> data = {
...   b'cart1': b'apples',
...   b'cart2': b'oranges',
...   b'cart3': b'grapes'
... }
>>> 
>>> new_data = { key.decode(): val.decode() for key, val in data.items() }
>>> 
>>> new_data
{'cart1': 'apples', 'cart2': 'oranges', 'cart3': 'grapes'}
>>>

To convert keys value pairs of type bytes in random order, use:

new_data = {
    key.decode() if isinstance(key, bytes) else key:
    val.decode() if isinstance(val, bytes) else val
    for key, val in data.items()
}

Example:

>>> data = {
...   b'cart1': 'apples',
...   'cart2': b'oranges',
...   b'cart3': b'grapes'
... }
>>> 
>>> new_data = {
...     key.decode() if isinstance(key, bytes) else key:
...     val.decode() if isinstance(val, bytes) else val
...     for key, val in data.items()
... }
>>> new_data
{'cart1': 'apples', 'cart2': 'oranges', 'cart3': 'grapes'}
>>>

Note: The above code will be better for simple data dictionary. But for complex dictionaries, I would prefer to use Guy Gangemi's code which is a modification of paxdiablo's answer:

def convert(data):
    if isinstance(data, bytes):  return data.decode()
    if isinstance(data, dict):   return dict(map(convert, data.items()))
    if isinstance(data, tuple):  return tuple(map(convert, data))
    if isinstance(data, list):   return list(map(convert, data))
    return data

Example:

>>> 
>>> def convert(data):
...     if isinstance(data, bytes):  return data.decode()
...     if isinstance(data, dict):   return dict(map(convert, data.items()))
...     if isinstance(data, tuple):  return tuple(map(convert, data))
...     if isinstance(data, list):   return list(map(convert, data))
...     return data
...
>>>
>>> data = {
...     b'fruits': {
...             b'cart1': b'apples',
...             b'cart2': 'oranges',
...             b'cart3': b'grapes',
...             b'cart4': (b'banana', 'pear'),
...             b'cart5': [b'kiwi', b'papaya']
...     },
...     'vegetables': {
...             'cart1': b'carrots',
...             b'cart2': None,
...             b'cart3': {},
...             b'cart4': False
...     }
... }
>>>
>>> convert(data)
{'fruits': {'cart1': 'apples', 'cart2': 'oranges', 'cart3': 'grapes', 'cart4': ('banana', 'pear'), 'cart5': ['kiwi', 'papaya']}, 'vegetables': {'cart1': 'carrots', 'cart2': None, 'cart3': {}, 'cart4': False}}
>>>
like image 31
jophine Avatar answered Sep 20 '22 09:09

jophine


Extending paxdiablo's answer to handle more use cases resulted in the following:

def convert(data):
  if isinstance(data, bytes):      return data.decode()
  if isinstance(data, (str, int)): return str(data)
  if isinstance(data, dict):       return dict(map(convert, data.items()))
  if isinstance(data, tuple):      return tuple(map(convert, data))
  if isinstance(data, list):       return list(map(convert, data))
  if isinstance(data, set):        return set(map(convert, data))

It becomes clear that the application of the map function is fairly consistent and we can generalise it.

def convert(data):
  data_type = type(data)

  if data_type == bytes: return data.decode()
  if data_type in (str, int): return str(data)

  if data_type == dict: data = data.items()
  return data_type(map(convert, data))
like image 27
Guy Gangemi Avatar answered Sep 21 '22 09:09

Guy Gangemi