How to remove deferred attribute of SQLAlchemy entity from memory?

Question

For example we have a table that stores LONGBLOB objects:

class MyEntity(_base):
    id = Column('ID', Integer, primary_key=True)
    metadata = Column('META', String(512), nullable=False)
    bigData = deferred(Column('BIG_DATA', LONGBLOB, nullable=False))

Column with LONGBLOB marked as deferred, i.e. it is loaded only on access to this attribute of particular object.

I need to query list of objects from the database and iterate over collection:

entities = dbSession.query(MyEntity).all()
for entity in entities:
    ...
    entity.bigData  # load BLOB from the database and do smth with data
    ...

In this loop all BLOBs will be loaded from database and memory will be exhausted. I need a way to clean memory at the end of each cycle. Expunge object from session and remove from memory entirely is not handy...

How to clean up particular attribute of the object (LONGBLOB) but do not remove object from the session?

Answer 1

Don't load bigData into the object. The deferred loading is already doing a query per iteration, so you can do that yourself without associating it with the MyEntity instance:

entities = dbSession.query(MyEntity).all()
for entity in entities:
    bigData = session.query(MyEntity.bigData).filter_by(id=entity.id).scalar()

bigData should be cleaned up as long as no references to it are kept around.