Closures over __class__

Question

I was recently stepping through the CPython source code, specifically viewing the symbol table entry for a class during compilation.

I bumped into the following entry for the typedef struct _symtable_entry structure:

[-- other entries --]
unsigned ste_needs_class_closure : 1; /* for class scopes, true if a
                                         closure over __class__
                                         should be created */
[-- other entries --]

I really can't seem to understand it and can't find an example of python code that actually sets ste_needs_class_closure == 1. Among other failed attempts, I have tried the following:

class foo:
    y = 30
    def __init__(self):
        self.x = 50
    def foobar(self):
        def barfoo():
            print(self.x)
            print(y)
        return barfoo

But even though it executes, the value of ste_needs_class_closure during execution is 0 and not 1 as I hoped it would be.

The function which actually alters this value is drop_class_free which doesn't help much. Unfortunately, it also doesn't have any comments complimenting it.

It is actually used in analyze_block with a comment of:

/* Check if any local variables must be converted to cell variables */

Which I can understand as a concept but cannot find an example where it happens.

I've tried searching the changelog for Python 3.4, the version in which this member first appeared but no references to it were found.

So, can anyone explain what is meant by closure over __class__ that is, when do local variables of a class get converted to cell variables? Ideally, an example that actually makes this behaviour visible during execution would be wonderfull.

Answer 1

Github’s blame view for that line of code shows us that it was added in this commit, which references Issue #12370: Prevent class bodies from interfering with the __class__ closure.

From the bug report, an example of the type of problem this was trying to fix is:

In Python 3 the following code prints False because the use of super() has caused the __class__ descriptor to be omitted from the class namespace. Remove the use of super and it prints True.

class X(object):
    def __init__(self):
        super().__init__()

    @property
    def __class__(self):
        return int

print (isinstance(X(), int))

(Note that this code uses the new super().)

Regarding the functionality of the patch, also from the bug report:

The patch basically causes the following class statement:

class C(A, B, metaclass=meta):
    def f(self):
        return __class__

To be compiled approximately like this:

def _outer_C(*__args__, **__kw__):
    class _inner_C(*__args__, **__kw__):
        def f(self):
            return __class__
    __class__ = _inner_C
    return _inner_C 
C = _outer_C(A, B, metaclass=meta)

… although some later discussion suggests that the handling of __args__ and __kw__ may have changed in the final patch.