json.loads allows duplicate keys in a dictionary, overwriting the first value

Question

>>> raw_post_data = request.raw_post_data >>> print raw_post_data {"group":{"groupId":"2", "groupName":"GroupName"}, "members":{"1":{"firstName":"fName","lastName":"LName","address":"address"},"1": {"firstName":"f_Name","lastName":"L_Name","address":"_address"}}} >>> create_request = json.loads(raw_post_data) >>> print create_request {u'group': {u'groupName': u'GroupName', u'groupId': u'2'}, u'members': {u'1': {u'lastName': u'L_Name', u'firstName': u'f_Name', u'address': u'_address'}}} 

As you can see members with key '1' is overwritten when I use json.dumps()

Is there any way to catch it as exception in python, saying found duplicate keys in request from client ?

Answer 1

The rfc 4627 for application/json media type recommends unique keys but it doesn't forbid them explicitly:

The names within an object SHOULD be unique.

From rfc 2119:

SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.

import json  def dict_raise_on_duplicates(ordered_pairs):     """Reject duplicate keys."""     d = {}     for k, v in ordered_pairs:         if k in d:            raise ValueError("duplicate key: %r" % (k,))         else:            d[k] = v     return d  json.loads(raw_post_data, object_pairs_hook=dict_raise_on_duplicates) # -> ValueError: duplicate key: u'1' 

Answer 2

This is a linter-fixed and type-annotated version of the answer by jfs. Issues highlighted by various linters were addressed. It is also modernized for Python 3.6+ to use f-strings.

import json from typing import Any, Dict, Hashable, List, Tuple  def raise_on_duplicate_keys(ordered_pairs: List[Tuple[Hashable, Any]]) -> Dict:     """Raise ValueError if a duplicate key exists in provided ordered list of pairs, otherwise return a dict."""     dict_out = {}     for key, val in ordered_pairs:         if key in dict_out:             raise ValueError(f'Duplicate key: {key}')         else:             dict_out[key] = val     return dict_out  json.loads('{"x": 1, "x": 2}', object_pairs_hook=raise_on_duplicate_keys) 

ordered_pairs above is a list of tuples, with each tuple having a key and a value. Refer to the docs for object_pairs_hook.