Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Lazy loading on column_property in SQLAlchemy

Say I have the following models:

class Department(Base):
    __tablename__ = 'departments'
    id = Column(Integer, primary_key=True)

class Employee(Base):
    __tablename__ = 'employees'
    id = Column(Integer, primary_key=True)
    department_id = Column(None, ForeignKey(Department.id), nullable=False)
    department = relationship(Department, backref=backref('employees'))

Sometimes, when I query departments, I would also like to fetch the number of employees they have. I can achieve this with a column_property, like so:

Department.employee_count = column_property(
    select([func.count(Employee.id)])
    .where(Employee.department_id == Department.id)
    .correlate_except(Employee)
)

Department.query.get(1).employee_count # Works

But then the count is always fetched via a subquery, even when I don't need it. Apparently I can't ask SQLAlchemy not to load this at query time, either:

Department.query.options(noload(Department.employee_count)).all()
# Exception: can't locate strategy for <class 'sqlalchemy.orm.properties.ColumnProperty'> (('lazy', 'noload'),)

I've also tried implementing this with a hybrid property instead of a column property:

class Department(Base):
    #...
    
    @hybrid_property
    def employee_count(self):
        return len(self.employees)

    @employee_count.expression
    def employee_count(cls):
        return (
            select([func.count(Employee.id)])
            .where(Employee.department_id == cls.id)
            .correlate_except(Employee)
        )

With no luck:

Department.query.options(joinedload('employee_count')).all()
# AttributeError: 'Select' object has no attribute 'property'

I know I can just query the count as a separate entity, but I need it often enough that I'd really prefer the convenience of having it as an attribute on the model. Is this even possible in SQLAlchemy?

Edit: To clarify, I want to avoid the N+1 problem and have the employee count get loaded in the same query as the departments, not in a separate query for each department.

like image 215
Sasha Chedygov Avatar asked Sep 13 '16 23:09

Sasha Chedygov


People also ask

What is lazy loading in sqlalchemy?

The loading of relationships falls into three categories; lazy loading, eager loading, and no loading. Lazy loading refers to objects are returned from a query without the related objects loaded at first.

What is lazy dynamic sqlalchemy?

lazy = 'dynamic': When querying with lazy = 'dynamic', however, a separate query gets generated for the related object. If you use the same query as 'select', it will return: You can see that it returns a sqlalchemy object instead of the city objects.

What is Backref in sqlalchemy?

The sqlalchemy backref is one of the type keywords and it passed as the separate argument parameters which has to be used in the ORM mapping objects. It mainly includes the event listener on the configuration attributes with both directions of the user datas through explicitly handling the database relationships.

What does all () do in sqlalchemy?

all() method. The Query object, when asked to return full entities, will deduplicate entries based on primary key, meaning if the same primary key value would appear in the results more than once, only one object of that primary key would be present. This does not apply to a query that is against individual columns.


1 Answers

The loading strategies that you tried are for relationships. The loading of a column_property is altered in the same way as normal columns, see Deferred Column Loading.

You can defer the loading of employee_count by default by passing deferred=True to column_property. When a column is deferred, a select statement is emitted when the property is accessed.

defer and undefer from sqlalchemy.orm allow this to be changed when constructing a query:

from sqlalchemy.orm import undefer
Department.query.options(undefer('employee_count')).all()
like image 65
RazerM Avatar answered Oct 16 '22 06:10

RazerM