Menu
I was working on a Flask project, getting some data from an API wrapper. The wrapper returned a generator object, so I print the values (for obj in gen_object: print obj) before passing it to Flask's render_template().
When requesting the page while printing the objects, the page is empty. But removing the for loop the page renders the generator object's content.
@app.route('/')
def front_page():
top_stories = r.get_front_page(limit=10)
# this for loop prevents the template from rendering the stories
for s in top_stories:
print s
return render_template('template.html', stories=top_stories)
346
Key takeaways: motivation and uses behind generators Generators are memory efficient since they only require memory for the one value they yield. Generators are lazy: they only yield values when explicitly asked. You can feed the output of a generator to the input of another generator to form data pipelines.
A Python generator is a function that produces a sequence of results. It works by maintaining its local state, so that the function can resume again exactly where it left off when called subsequent times. Thus, you can think of a generator as something like a powerful iterator.
A generator is a special type of function which does not return a single value, instead, it returns an iterator object with a sequence of values.
Generators are memory-efficient ways of processing huge datasets. They process the data incrementally and do not allocate memory to all the results at the same time. They really come in handy when implementing data science pipelines for huge datasets in a resource-constrained environment (in terms of RAM).
Yes, generators are ment to be consumed once. Each time we iterate a generator we ask it to give us another value, and if there's no more values to give the StopIteration exception is thrown which would stop the iteration. There's no way for the generator to know that we want to iterate it again without cloning it.
As long as the records can fit comfortably in memory, I would use a list instead:
top_stories = list(top_stories)
That way you can iterate top_stories multiple times.
There's a function called itertools.tee to copy an iterator that also could help you but it's sometimes slower than just using a list. Reference: http://docs.python.org/library/itertools.html?highlight=itertools.tee#itertools.tee
125
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
