Why does using `arg=None` fix Python's mutable default argument issue?

Question

I'm at the point in learning Python where I'm dealing with the Mutable Default Argument problem.

# BAD: if `a_list` is not passed in, the default will wrongly retain its contents between successive function calls def bad_append(new_item, a_list=[]):     a_list.append(new_item)     return a_list  # GOOD: if `a_list` is not passed in, the default will always correctly be [] def good_append(new_item, a_list=None):     if a_list is None:         a_list = []     a_list.append(new_item)     return a_list 

I understand that a_list is initialized only when the def statement is first encountered, and that's why subsequent calls of bad_append use the same list object.

What I don't understand is why good_append works any different. It looks like a_list would still be initialized only once; therefore, the if statement would only be true on the first invocation of the function, meaning a_list would only get reset to [] on the first invocation, meaning it would still accumulate all past new_item values and still be buggy.

Why isn't it? What concept am I missing? How does a_list get wiped clean every time good_append runs?

Answer 1

It looks like a_list would still be initialized only once

"initialization" is not something that happens to variables in Python, because variables in Python are just names. "initialization" only happens to objects, and it's done via the class' __init__ method.

When you write a = 0, that is an assignment. That is saying "a shall refer to the object that is described by the expression 0". It is not initialization; a can name anything else of any type at any later time, and that happens as a result of assigning something else to a. Assignment is just assignment. The first one is not special.

When you write def good_append(new_item, a_list=None), that is not "initializing" a_list. It is setting up an internal reference to an object, the result of evaluating None, so that when good_append is called without a second parameter, that object is automatically assigned to a_list.

meaning a_list would only get reset to [] on the first invocation

No, a_list gets set to [] any time that a_list is None to begin with. That is, when either None is passed explicitly, or the argument is omitted.

The problem with [] occurs because the expression [] is only evaluated once in this context. When the function is compiled, [] is evaluated, a specific list object is created - that happens to be empty to start - and that object is used as the default.

How does a_list get wiped clean every time good_append runs?

It doesn't. It doesn't need to be.

You know how the problem is described as being with "mutable default arguments"?

None is not mutable.

The problem occurs when you modify the object that the parameter has as a default.

a_list = [] does not modify whatever object a_list previously referred to. It cannot; arbitrary objects cannot magically transform in-place into empty lists. a_list = [] means "a_list shall stop referring to what it previously referred to, and start referring to []". The previously-referred-to object is unchanged.

When the function is compiled, and one of the arguments has a default value, that value - an object - gets baked into the function (which is also, itself, an object!). When you write code that mutates an object, the object mutates. If the object being referred to happens to be the object baked into the function, it still mutates.

But you cannot mutate None. It is immutable.

You can mutate []. It is a list, and lists are mutable. Appending an item to a list mutates the list.

Answer 2

The default value of a_list (or any other default value, for that matter) is stored in the function's interiors once it has been initialized and thus can be modified in any way:

>>> def f(x=[]): return x ... >>> f.func_defaults ([],) >>> f.func_defaults[0] is f() True 

resp. for Python 3:

>>> def f(x=[]): return x ... >>> f.__defaults__ ([],) >>> f.__defaults__[0] is f() True 

So the value in func_defaults is the same which is as well known inside function (and returned in my example in order to access it from outside.

In other words, what happens when calling f() is an implicit x = f.func_defaults[0]. If that object is modified subsequently, you'll keep that modification.

In contrast, an assignment inside the function gets always a new []. Any modification will last until the last reference to that [] has gone; on the next function call, a new [] is created.

In order words again, it is not true that [] gets the same object on every execution, but it is (in the case of default argument) only executed once and then preserved.