Python for-loop look-ahead

Question

I have a python for loop, in which I need to look ahead one item to see if an action needs to be performed before processing.

for line in file:
    if the start of the next line == "0":
        perform pre-processing
        ...
    continue with normal processing
    ...

Is there any easy way to do this in python? My current approach is to buffer the file to an array, however this is not ideal as the file is rather large.

Answer 1

you can get any iterable to prefetch next item with this recipe:

from itertools import tee, islice, izip_longest
def get_next(some_iterable, window=1):
    items, nexts = tee(some_iterable, 2)
    nexts = islice(nexts, window, None)
    return izip_longest(items, nexts)

Example usage:

for line, next_line in get_next(myfile):
    if next_line and next_line.startswith("0"):
        ... do stuff

The code allows you to pass the window parameter as a larger value, if you want to look 2 or more lines ahead.

Answer 2

Along the lines of nosklo's answer, I tend to use the following pattern:

The function pairwise from the excellent itertools recipes is ideal for this:

from itertools import tee

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return izip(a, b)

Using it in your code gets us:

for line, next_line in pairwise(file):
    if next_line.startswith("0"):
        pass #perform pre-processing
        #...
    pass #continue with normal processing

Generally, for this type of processing (lookahead in the iterable), I tend to use a window function. Pairwise is a special case of a window of size 2.