When and why to use self.__dict__ instead of self.variable

Question

I'm trying to understand some code which is using this class below:

class Base(object):

    def __init__(self, **kwargs):
        self.client = kwargs.get('client')
        self.request = kwargs.get('request')
    ...

    def to_dict(self):
        data = dict()

        for key in iter(self.__dict__): # <------------------------ this
            if key in ('client', 'request'):
                continue

            value = self.__dict__[key]
            if value is not None:
                if hasattr(value, 'to_dict'):
                    data[key] = value.to_dict()
                else:
                    data[key] = value
        return data

I understand that it gets keyword arguments passed to the Base class like for example, Base(client="foo", request="bar").

My confusion is, why is it using self.__dict__ which turns variables inside __init__ to a dict (e.g {"client": "foo", "request": "bar"}) instead of just calling them by self.client & self.request inside other methods? When and why I should use self.__dict__ instead?

Answer 1

Almost all of the time, you shouldn't use self.__dict__.

If you're accessing an attribute like self.client, i.e. the attribute name is known and fixed, then the only difference between that and self.__dict__['client'] is that the latter won't look up the attribute on the class if it's missing on the instance. There is very rarely any reason to do this, but the difference is demonstrated below:

>>> class A:
...     b = 3 # class attribute, not an instance attribute
... 
>>> A.b # the class has this attribute
3
>>> a = A()
>>> a.b # the instance doesn't have this attribute, fallback to the class
3
>>> a.__dict__['b'] # the instance doesn't have this attribute, but no fallback
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'

The main use-case for self.__dict__ is when you don't want to access a fixed, known attribute name. In almost all code, you always know which attribute you want to access; and if you do need to look something up dynamically using an unknown string, you should create a dictionary yourself, and write self.that_dict[key] instead of self.__dict__[key].

So the only times you should really use __dict__ is when you are writing code which needs to work regardless of which attributes the instance might have; i.e. you specifically want code which will work even if you change the class's structure or its attribute names, or code which will work across multiple classes with different structures. I'll show one example below.

The __repr__ method

The __repr__ method is meant to return a string representing the instance, for the programmer's convenience when using a REPL. For debugging/testing purposes this string usually contains information about the object's state. Here's a common way to implement it:

class Foo:
    def __init__(self, foo, bar, baz):
        self.foo = foo
        self.bar = bar
        self.baz = baz

    def __repr__(self):
        return 'Foo({!r}, {!r}, {!r})'.format(self.foo, self.bar, self.baz)

This means if you write obj = Foo(1, 'y', True) to create an instance, then repr(obj) will be the string "Foo(1, 'y', True)", which is convenient because it shows the instance's entire state, and also the string itself is Python code which creates an instance with the same state.

But there are a few issues with the above implementation: we have to change it if the class's attributes change, it won't give useful results for instances of subclasses, and we have to write lots of similar code for different classes with different attributes. If we use __dict__ instead, we can solve all of those problems:

    def __repr__(self):
        return '{}({})'.format(
            self.__class__.__name__,
            ', '.join('{}={!r}'.format(k, v) for k, v in self.__dict__.items())
        )

Now repr(obj) will be Foo(foo=1, bar='y', baz=True), which also shows the instance's entire state, and is also executable Python code. This generalised __repr__ method will still work if the structure of Foo changes, it can be shared between multiple classes via inheritance, and it returns executable Python code for any class whose attributes are accepted as keyword arguments by __init__.

Answer 2

__dict__ holds all of the variables in the class. Take the following class:

class A():
    def __init__(self, foo):
        self.foo = foo

    def new_var(self, bar):
        self.bar = bar

Then in this case, notice:

a = A('var1')
print(a.__dict__) # {'foo': 'var1'}

b = A('var1')
b.new_var('var2')
b.foobar = 'var3'
print(b.__dict__) # {'foo': 'var1', 'bar': 'var2', 'foobar': 'var3'}

In your case you could do either or. __dict__ is a great way to grab all of the variables that are part of that class at the current instance in which it is called. You can check out the documentation on __dict__ here.