I have a custom container class in Python 2.7, and everything works as expected except if I pass try to expand an instance as **kwargs
for a function:
cm = ChainableMap({'a': 1})
cm['b'] = 2
assert cm == {'a': 1, 'b': 2} # Is fine
def check_kwargs(**kwargs):
assert kwargs == {'a': 1, 'b': 2}
check_kwargs(**cm) # Raises AssertionError
I've overridden __getitem__
, __iter__
, iterkeys
, keys
, items
, and iteritems
, (and __eq__
and __repr__
) yet none of them seem to be involved in the expansion as **kwargs
, what am I doing wrong?
Edit - The working updated source that now inherits from MutableMapping and adds the missing methods:
from itertools import chain
from collections import MutableMapping
class ChainableMap(MutableMapping):
"""
A mapping object with a delegation chain similar to JS object prototypes::
>>> parent = {'a': 1}
>>> child = ChainableMap(parent)
>>> child.parent is parent
True
Failed lookups delegate up the chain to self.parent::
>>> 'a' in child
True
>>> child['a']
1
But modifications will only affect the child::
>>> child['b'] = 2
>>> child.keys()
['a', 'b']
>>> parent.keys()
['a']
>>> child['a'] = 10
>>> parent['a']
1
Changes in the parent are also reflected in the child::
>>> parent['c'] = 3
>>> sorted(child.keys())
['a', 'b', 'c']
>>> expect = {'a': 10, 'b': 2, 'c': 3}
>>> assert child == expect, "%s != %s" % (child, expect)
Unless the child is already masking out a certain key::
>>> del parent['a']
>>> parent.keys()
['c']
>>> assert child == expect, "%s != %s" % (child, expect)
However, this doesn't work::
>>> def print_sorted(**kwargs):
... for k in sorted(kwargs.keys()):
... print "%r=%r" % (k, kwargs[k])
>>> child['c'] == 3
True
>>> print_sorted(**child)
'a'=10
'b'=2
'c'=3
"""
__slots__ = ('_', 'parent')
def __init__(self, parent, **data):
self.parent = parent
self._ = data
def __getitem__(self, key):
try:
return self._[key]
except KeyError:
return self.parent[key]
def __iter__(self):
return self.iterkeys()
def __setitem__(self, key, val):
self._[key] = val
def __delitem__(self, key):
del self._[key]
def __len__(self):
return len(self.keys())
def keys(self, own=False):
return list(self.iterkeys(own))
def items(self, own=False):
return list(self.iteritems(own))
def iterkeys(self, own=False):
if own:
for k in self._.iterkeys():
yield k
return
yielded = set([])
for k in chain(self.parent.iterkeys(), self._.iterkeys()):
if k in yielded:
continue
yield k
yielded.add(k)
def iteritems(self, own=False):
for k in self.iterkeys(own):
yield k, self[k]
def __eq__(self, other):
return sorted(self.iteritems()) == sorted(other.iteritems())
def __repr__(self):
return dict(self.iteritems()).__repr__()
def __contains__(self, key):
return key in self._ or key in self.parent
def containing(self, key):
"""
Return the ancestor that directly contains ``key``
>>> p2 = {'a', 2}
>>> p1 = ChainableMap(p2)
>>> c = ChainableMap(p1)
>>> c.containing('a') is p2
True
"""
if key in self._:
return self
elif hasattr(self.parent, 'containing'):
return self.parent.containing(key)
elif key in self.parent:
return self.parent
def get(self, key, default=None):
"""
>>> c = ChainableMap({'a': 1})
>>> c.get('a')
1
>>> c.get('b', 'default')
'default'
"""
if key in self:
return self[key]
else:
return default
def pushdown(self, top):
"""
Pushes a new mapping onto the top of the delegation chain:
>>> parent = {'a': 10}
>>> child = ChainableMap(parent)
>>> top = {'a': 'apple', 'b': 'beer', 'c': 'cheese'}
>>> child.pushdown(top)
>>> assert child == top
This creates a new ChainableMap with the contents of ``child`` and makes it
the new parent (the old parent becomes the grandparent):
>>> child.parent.parent is parent
True
>>> del child['a']
>>> child['a'] == 10
True
"""
old = ChainableMap(self.parent)
for k, v in self.items(True):
old[k] = v
del self[k]
self.parent = old
for k, v in top.iteritems():
self[k] = v
Both Python *args and **kwargs let you pass a variable number of arguments into a function. *args arguments have no keywords whereas **kwargs arguments each are associated with a keyword. Traditionally, when you're working with functions in Python, you need to directly state the arguments the function will accept.
*args allows us to pass a variable number of non-keyword arguments to a Python function. In the function, we should use an asterisk ( * ) before the parameter name to pass a variable number of arguments.
**kwargs works just like *args , but instead of accepting positional arguments it accepts keyword (or named) arguments. Take the following example: # concatenate.py def concatenate(**kwargs): result = "" # Iterating over the Python kwargs dictionary for arg in kwargs.
First of all, let me tell you that it is not necessary to write *args or **kwargs. Only the * (asterisk) is necessary. You could have also written *var and **vars. Writing *args and **kwargs is just a convention.
When creating a keyword argument dictionary, the behavior is the same as passing your object into the dict()
initializer, which results in the dict {'b': 2}
for your cm
object:
>>> cm = ChainableMap({'a': 1})
>>> cm['b'] = 2
>>> dict(cm)
{'b': 2}
A more detailed explanation of why this is the case is below, but the summary is that your mapping is converted to a Python dictionary in C code which does some optimization if the argument is itself another dict, by bypassing the Python function calls and inspecting the underlying C object directly.
There are a few ways to approach the solution for this, either make sure that the underlying dict contains everything you want, or stop inheriting from dict (which will require other changes as well, at the very least a __setitem__
method).
edit: It sounds like BrenBarn's suggestion to inherit from collections.MutableMapping
instead of dict
did the trick.
You could accomplish the first method pretty simply by just adding self.update(parent)
to ChainableMap.__init__()
, but I'm not sure if that will cause other side effects to the behavior of your class.
Explanation of why dict(cm)
gives {'b': 2}
:
Check out the following CPython code for the dict object:
http://hg.python.org/releasing/2.7.3/file/7bb96963d067/Objects/dictobject.c#l1522
When dict(cm)
is called (and when keyword arguments are unpacked), the PyDict_Merge
function is called with cm
as the b
parameter. Because ChainableMap inherits from dict, the if statement at line 1539 is entered:
if (PyDict_Check(b)) {
other = (PyDictObject*)b;
...
From there on, items from other
are added to the new dict that is being created by accessing the C object directly, which bypasses all of the methods that you overwrote.
This means that any items in a ChainableMap instance that are accessed through the parent
attribute will not be added to the new dictionary created by dict()
or keyword argument unpacking.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With