Is there a way to remember the position in a python iterator?

Question

I would like to iterate over an iterable object (let's say, a list) and leave at some point remembering the position where I left off to continue the next time an iterator for that object is called.

Something like:

for val in list:
   do_stuff(val)
   if some_condition:
       break

do_stuff()

for val in list:
    continue_doing_stuff(val)

Speed matters and the list considered is quite large. So saving the object and iterating again through the whole list until the saved element is found is not an option. Is this possible without writing an explicit iterator class for the list?

Answer 1

The __iter__ method is called when you enter a for loop with an object, returning an iterator. We usually don't keep a name pointing to the iterator, but if we do, we can stop the iterating, do something else, and then resume the iterating.

The best way to get the iterator object is to use the builtin iter function:

a_list = ['a', 'b', 'c', 'd']
iter_list = iter(a_list)

for val in iter_list:
   print(val)                # do_stuff(val)
   if val == 'b':            # some_condition!
       break

print('taking a break')      # do_stuff()

for val in iter_list:
    print(val)               # continue_doing_stuff(val)

shows:

a
b
taking a break
c
d

iter(obj) just returns the result of obj.__iter__(), which should be an iterator implementing a .__next__() method.

That __next__ method is called for each iteration, returning the object (in this case, a character.)

If you want to call the __next__ method yourself instead of having it called by the for loop, you should use the builtin next function:

a_list = ['a', 'b', 'c', 'd']
iter_list = iter(a_list)
print(next(iter_list))       # do_stuff(val)
print(next(iter_list))
print('taking a break')      # do_stuff()
print(next(iter_list))       # continue_doing_stuff(val)
print(next(iter_list))

prints:

a
b
taking a break
c
d

Answer 2

You can use a generator to do this

def get_next(iterator):
    for item in iterator:
        yield item

my_list_iterator = get_next(my_list)

for val in my_list_iterator:
    do_stuff(val)
    if some_condition:
        break

do_stuff()

for val in my_list_iterator:
    continue_doing_stuff(val)