How to clone a Python generator object?

Question

Consider this scenario:

#!/usr/bin/env python # -*- coding: utf-8 -*- import os  walk = os.walk('/home')  for root, dirs, files in walk:     for pathname in dirs+files:         print os.path.join(root, pathname)  for root, dirs, files in walk:     for pathname in dirs+files:         print os.path.join(root, pathname) 

I know that this example is kinda redundant, but you should consider that we need to use the same walk data more than once. I've a benchmark scenario and the use of same walk data is mandatory to get helpful results.

I've tried walk2 = walk to clone and use in the second iteration, but it didn't work. The question is... How can I copy it? Is it ever possible?

Thank you in advance.

Answer 1

You can use itertools.tee():

walk, walk2 = itertools.tee(walk) 

Note that this might "need significant extra storage", as the documentation points out.

Answer 2

If you know you are going to iterate through the whole generator for every usage, you will probably get the best performance by unrolling the generator to a list and using the list multiple times.

walk = list(os.walk('/home'))