Periodically delete long dicts / lists in Python good practice?

Question

I have been writing a long script that occasionally builds large dicts and/or lists, and I was wondering if performance could be improved by deleting them with del when I'm finished using them. Or is it normal practice to leave these objects hanging around to be taken care of by garbage collection? What is the best practice here? Thanks.

Answer 1

del is not equivalent to free(3). It doesn't force Python to free memory. It may not end up freeing memory at all. You should avoid associating it with memory usage entirely.

The only thing del does is delete a name from its scope. (Or delete an item from a collection, or delete an attribute. But I assume that's not what you're talking about here.)

Effectively, this:

del foo

Is equivalent to this:

del LOCAL_SCOPE['foo']

So this doesn't free any memory:

massive_list = list(range(1000000))
same_massive_list = massive_list
del massive_list

...because all it does is remove the name massive_list. The underlying object still has another name, same_massive_list, so it doesn't go away. del isn't a secret hook for controlling Python's memory management; it's just one of several ways to ask Python to invoke its memory management.

(By the way, CPython is refcounted + cycle-collected, not garbage-collected. Objects are immediately freed as soon as the last reference to them disappears. Garbage does not lie around waiting to be cleaned up. Of course, other implementations do different things; PyPy, for example, is garbage collected.)

Now, if the name you use is the only name for the list/dict/whatever, then del will certainly cause its refcount to drop to zero, so it'll be freed. But, since del's semantics are really about removing names, I wouldn't use it in this case. I'd just let the variable drop out of scope (if practical), or reassign the name to a blank list, or None, or whatever makes sense for your program. You can even empty the list in-place, which will work even if there are several names for the same list:

foo = list(range(1000000))
bar = foo
foo[:] = []
# Both `bar` and `foo` still refer to the original list, but now it's empty

You can do the same thing to a dict with d.clear().

The only place I'd use del on a name is in a class or module scope, where I temporarily need some helper value but really really don't want to expose it as part of the API. This is really rare, but it's the only case I've run into where I actually want the semantics of "remove this name".

Answer 2

del is usually not necessary.

In CPython, objects go away when there are no references to them. del deletes only one reference to the object; if there are still other references to the object, it will stay around, and del has really done nothing to improve your memory situation.

Conversely, simply reassigning the variable to a different object (say, creating a new empty list at the top of a loop) will also end up with one less reference to the object, implicitly doing the same as del. If there are no other references to the object at this point, it will be freed immediately.

Also remember that local variables go away when the function returns, so you don't need to explicitly del any names you've defined in a function.

An exception is circular references, where two (or more) objects reference each other but none is actually reachable through a name; Python periodically garbage-collects these, but you can free them up faster if you break the circle when you're done with the objects. (This may require as little as one del!) Circumstances in which this is useful are probably pretty rare, though.

In IronPyothon (which runs on the .NET CLR) or Jython (which runs on the JVM), optimal memory management strategies may differ, since the garbage collector of the underlying VM is used.