SQLAlchemy custom local attribute in model

Question

I have a simple model defined and additionally added some non-SQL related attributes to the class.

If the attribute is an int or string type things are working. But if it is a dictionary or list type then magically the same list is used over all different instances of the model-class.

Example:

# Sample model class
class TestClass(Base):
   # SQL Mappings   
   __tablename__ = 'test1'

   pid = Column("id", Integer, primary_key=True)
   name = Column('name', String)

   # Non SQL related attributes
   works_var = 0
   works_not_var = []
   # ...

Somewhere else after a query working on retrieved TestClass instance

my_test_class.works_not_var.append("testval1")
my_test_class2.works_not_var.append("testval2")

Somehow both appended values end up being in the same list:

print id(my_test_class.works_not_var)
print id(my_test_class2.works_not.var)

Id's are the same. But it works for "works_var". Id's there are different.

Answer 1

This is absolutely not related to SQLAlchemy and is standard Python behaviour. The reason for the problem is related how Python handles types and when things are parsed. Consider this funny example:

class A(object):
    print "Hello"
print "A is now defined"
a = A()
print "I now have an instance of A"

This is of course totally useless but observe, when executed, when the order of prints:

Hello
A is now defined
I now have an instance of A

Would you have expected this?

Simple solution

# Sample model class
class TestClass(Base):
   # SQL Mappings   
   __tablename__ = 'test1'

   pid = Column("id", Integer, primary_key=True)
   name = Column('name', String)
  # ...

  def __init__(self):
    self.works_var = 0
    self.works_not_var = []

Rule of Thumb: Place default parameters in __init__ not on the class level.

Explanation

A more thorough explanation might be in order: Why does the problem occur? I will not go into detail on how Python handles variables in general. There is a neat explanation in eevee's Python FAQ: Passing article. Additionally, there is a nice explanation here: Other languages have "variables".

Armed with that knowledge and with the example from above we now know when the statement works_not_var = [] gets executed: The moment it is imported (or the script is started). And we know also see why this is a problem: An object like list is mutable and Python will not move its "tag" when you change it: Instead you created an instance variable. More commonly this problem is noticed (and easier to explain) on functions default arguments. Consider this SO question: “Least Astonishment” in Python: The Mutable Default Argument. It explains very good how where this comes from. In a concise example:

def f(a=[]):
    if len(a) == 0:
        print "Oh no, list is empty"
        a.append(1)

f()
print "Function executed first time"
f()
print "Function executed second time"

And the output:

Oh no, list is empty
Function executed first time
Function executed second time

The list is created on parsing them, not execution time. Another example of how this can fail and create stupid problems:

from datetime import datetime
from time import sleep

def f(time=datetime.now()):
    print time

f()
sleep(1)
f()
f(datetime.now())

So you created a function that gets a time that defaults to the current time. Well, not so much. It defaults to the time the program was started but not the current time. If you run it you will get:

2013-08-20 16:14:29.037069
2013-08-20 16:14:29.037069
2013-08-20 16:14:30.038302

But you would expect that the second and third time would be almost equal and not different at the "second" level. The problem again: The default parameter datetime.now() is executed when the function is parsed not executed.

The solution

For this there also exists an easy solution (though I might say I don't find it as pretty as I'd like):

def f(now=None):
    if now is None:
        now = datetime.now()

I hope this explanation helps.