Psycopg2 cursor.execute return a generator

Question

Say i have only 1GB of memory and 1 TB of hard disk space.

This is my code and i am using a postgres database.

import psycopg2

try:
   db = psycopg2.connect("database parameters")
   conn = db.cursor()
   conn.execute(query) 

   #At this point, i am running 
   for row in conn:

for this case, I guess it is safe to assume that conn is a generator as i cannot seem to find a definitive answer online and i cannot try it on my environment as i cannot afford the system to crash.

I am expecting this query to return data in excess of 100 GB

I am using python 2.7 and psycopg2 library

Answer 1

If you use an anonymous cursor, which you are doing in your example, then the entire query result will be read into memory.

If you use a named cursor then it will read from the server in chunks as it loops over the data.