why does a call to locals() add a reference?

Question

I don't understand the below behavior.

• How does locals() result in a new reference?
• Why doesn't gc.collect remove it? I didn't assign the result of locals() anywhere.

x

import gc

from sys import getrefcount

def trivial(x): return x

def demo(x):
    print getrefcount(x)
    x = trivial(x)
    print getrefcount(x)
    locals()
    print getrefcount(x)
    gc.collect()
    print getrefcount(x)


demo(object())

The output is:

$ python demo.py 
3
3
4
4

Answer 1

This has to do with 'fast locals' which are stored as a matched pair of tuples for fast integer indexing (one for names f->f_code->co_varnames, one for values f->f_localsplus). When locals() is called, the fast-locals are converted into a standard dict and tacked onto the frame structure. The relevant bits of the cpython code are below.

This is the implementing function for locals(). It does little more than call PyEval_GetLocals.

static PyObject *
builtin_locals(PyObject *self)
{
    PyObject *d;

    d = PyEval_GetLocals();
    Py_XINCREF(d);
    return d;
}   

In turn PyEval_GetLocals does little more than call PyFrame_FastToLocals.

PyObject *
PyEval_GetLocals(void)
{   
    PyFrameObject *current_frame = PyEval_GetFrame();
    if (current_frame == NULL)
        return NULL;
    PyFrame_FastToLocals(current_frame);
    return current_frame->f_locals;
}

This is the bit that allocates a plain-old dictionary for the frame's local variables and stuffs any "fast" variables into it. Since the new dict is tacked onto the frame structure (as f->f_locals), any "fast" variables get an extra reference upon a call to locals().

void
PyFrame_FastToLocals(PyFrameObject *f)
{
    /* Merge fast locals into f->f_locals */
    PyObject *locals, *map;
    PyObject **fast;
    PyObject *error_type, *error_value, *error_traceback;
    PyCodeObject *co;
    Py_ssize_t j;
    int ncells, nfreevars;
    if (f == NULL)
        return;
    locals = f->f_locals;
    if (locals == NULL) {
        /* This is the dict that holds the new, additional reference! */
        locals = f->f_locals = PyDict_New();  
        if (locals == NULL) {
            PyErr_Clear(); /* Can't report it :-( */
            return;
        }
    }
    co = f->f_code;
    map = co->co_varnames;
    if (!PyTuple_Check(map))
        return;
    PyErr_Fetch(&error_type, &error_value, &error_traceback);
    fast = f->f_localsplus;
    j = PyTuple_GET_SIZE(map);
    if (j > co->co_nlocals)
        j = co->co_nlocals;
    if (co->co_nlocals)
        map_to_dict(map, j, locals, fast, 0);
    ncells = PyTuple_GET_SIZE(co->co_cellvars);
    nfreevars = PyTuple_GET_SIZE(co->co_freevars);
    if (ncells || nfreevars) {
        map_to_dict(co->co_cellvars, ncells,
                    locals, fast + co->co_nlocals, 1);
        /* If the namespace is unoptimized, then one of the
           following cases applies:
           1. It does not contain free variables, because it
              uses import * or is a top-level namespace.
           2. It is a class namespace.
           We don't want to accidentally copy free variables
           into the locals dict used by the class.
        */
        if (co->co_flags & CO_OPTIMIZED) {
            map_to_dict(co->co_freevars, nfreevars,
                        locals, fast + co->co_nlocals + ncells, 1);
        }
    }
    PyErr_Restore(error_type, error_value, error_traceback);
}

Answer 2

I added a few prints to your demo code :

#! /usr/bin/python

import gc

from sys import getrefcount

def trivial(x): return x

def demo(x):
    print getrefcount(x)
    x = trivial(x)
    print getrefcount(x)
    print id(locals())
    print getrefcount(x)
    print gc.collect(), "collected"
    print id(locals())
    print getrefcount(x)


demo(object())

The output is then (on my machine):

3
3
12168320
4
0 collected
12168320
4

locals() actually creates a dict containing a ref on x, thus the ref inc. gc.collect() does not collect the locals dict, you can see it by printing the id, it's the same object returned twice, it is somehow memoized for this frame, thus not collected.