SQLAlchemy and going through a large result set [duplicate]

Question

I need to read data from all of the rows of a large table, but I don't want to pull all of the data into memory at one time. Is there a SQLAlchemy function that will handle paging? That is, pull several rows into memory and then fetch more when necessary.

I understand you can do this with limit and offset as this article suggests, but I'd rather not handle that if I don't have to.

Answer 1

If you are using Flask-SqlAlchemy, see the paginate method of query. paginate offers several method to simplify pagination.

record_query = Record.query.paginate(page, per_page, False) total = record_query.total record_items = record_query.items 

First page should be 1 otherwise the .total returns exception divided by zero