Reference class variable in a comprehension of another class variable

Question

This may be a simple question, but I'm having trouble making a unique search for it.

I have a class that defines a static dictionary, then attempts to define a subset of that dictionary, also statically.

So, as a toy example:

class example(object): 
    first_d = {1:1,2:2,3:3,4:4} 
    second_d = dict((k,first_d[k]) for k in (2,3))

This produces NameError: global name 'first_d' is not defined

How should I be making this reference? It seems this pattern works in other cases, eg:

class example2(object):
    first = 1
    second = first + 1

Answer 1

A basic list comprehension has the following syntax

[expression for var in iterable]

When a list comprehension occurs inside a class, the attributes of the class can be used in iterable. This is true in Python2 and Python3.

However, the attributes of the class can be used (i.e. accessed) in expression in Python2 but not in Python3.

The story is a bit different for generator expressions:

(expression for var in iterable)

While the class attributes can still be accessed from iterable, the class attributes are not accessible from expression. (This is true for Python2 and Python3).

This can all be summarized as follows:

                             Python2      Python3
Can access class attributes
--------------------------------------------------
list comp. iterable                Y            Y
list comp. expression              Y            N
gen expr. iterable                 Y            Y
gen expr. expression               N            N
dict comp. iterable                Y            Y
dict comp. expression              N            N

(Dict comprehensions behave the same as generator expressions in this respect.)

---

Now how does this relate to your question:

In your example,

second_d = dict((k,first_d[k]) for k in (2,3))

a NameError occurs because first_d is not accessible from the expression part of a generator expression.

A workaround for Python2 would be to change the generator expression to a list comprehension:

second_d = dict([(k,first_d[k]) for k in (2,3)])

However, I don't find this a very comfortable solution since this code will fail in Python3.

You could do as Joel Cornett suggests:

second_d = {k: v for k, v in first_d.items() if k in (2, 3)}

since this uses first_d in the iterable rather than the expression part of the dict comprehension. But this may loop through many more items than necessary if first_d contains many items. Neverthess, this solution might be just fine if first_d is small.

In general, you can avoid this problem by defining a helper function which can be defined inside or outside the class:

def partial_dict(dct, keys):
    return {k:dct[k] for k in keys}

class Example(object):
    first_d = {1:1,2:2,3:3,4:4}
    second_d = partial_dict(first_d, (2,3))

class Example2(object):
    a = [1,2,3,4,5]
    b = [2,4]
    def myfunc(A, B):
        return [x for x in A if x not in B]
    c = myfunc(a, b)

print(Example().second_d)
# {2: 2, 3: 3}

print(Example2().c)
# [1, 3, 5]

Functions work because they define a local scope and variables in this local scope can be accessed from within the dict comprehension.

This was explained here, but I am not entirely comfortable with this since it does not explain why the expression part behaves differently than the iterable part of a list comprehension, generator expression or dict comprehension.

Thus I can not explain (completely) why Python behaves this way, only that this is the way it appears to behave.

Answer 2

It's a bit kludgy, but you could try this:

class test(object):
    pass

test.first = {1:1, 2:2, 3:3, 4:4}
test.second = dict((k, test.first[k]) for k in (2,3))

...and then:

>>> test.first
{1: 1, 2: 2, 3: 3, 4: 4}
>>> test.second
{2: 2, 3: 3}

>>> t = test()
>>> t.first
{1: 1, 2: 2, 3: 3, 4: 4}
>>> t.second
{2: 2, 3: 3}

>>> test.first[5] = 5
>>> t.first
{1: 1, 2: 2, 3: 3, 4: 4, 5: 5}