Python process doesn't release memory

Question

I have got a dict with following data,

my_dict = {'item_1': <generator object continuation_stripper at 0xda78640>,
           'only_display_attrs': <generator object continuation_stripper at 0xda78690>,
           'item_2': <generator object continuation_stripper at 0xda786e0>}

And I am trying to loop over a dict value only_display_attrs , this dict value is a generator object.

this is my code,

for item in my_dict['only_display_attrs']:
    yield i

Note: only_display_attrs is a SQLAlchemy generator object.

This loop consumes upto 1100 mb of memory for 30,000 rows of data and doesn't release the memory till the process is killed manually.

Can anyone help me understand the reason behind this.

Answer 1

The relevant bit is what you do with the values that are yielded. If you keep them around, then the memory will never be released. But in the code displayed, no memory is being used up (unless there is some mistake in the continuation_stripper code).