Iterating over PyoDBC result without fetchall()

Question

I'm trying to process a very large query with pyodbc and I need to iterate over the rows without loading them all at once with fetchall().

Is there a good and principled way to do this?

Answer 1

According to official documentation the cursor is apparently an iterator. Therefore, you shouldn't need to create a custom iterator/generator.

If you are going to process the rows one at a time, you can use the cursor itself as an iterator:

cursor.execute("select user_id, user_name from users"):
for row in cursor:
    print(row.user_id, row.user_name)

Answer 2

Sure - use a while loop with fetchone.

http://code.google.com/p/pyodbc/wiki/Cursor#fetchone

row = cursor.fetchone()
while row is not None:
    # do something
    row = cursor.fetchone()

Answer 3

you could also use cursor.fetchmany() if you want to batch up the fetches (defaults to 1 if you don't override it)

http://code.google.com/p/pyodbc/wiki/Cursor#fetchmany