for loop save to array but skip saving elements

Question

Basically, I want a fancy oneliner that doesn't read all of the files I'm looking at into memory, but still processes them all, and saves a nice sample of them.

The oneliner I would like to do is:

def foo(findex):
    return [bar(line) for line in findex] # but skip every nth term

But I would like to be able to not save every nth line in that. i.e., I still want it to run (for byte position purposes), but I don't want to save the image, because I don't have enough memory for that.

So, if the output of bar(line) is 1,2,3,4,5,6,... I would like it to still run on 1,2,3,4,5,6,... but I would like the return value to be [1,3,5,7,9,...] or something of the sort.

Answer 1

use enumerate to get the index, and a filter using modulo to take every other line:

return [bar(line) for i,line in enumerate(findex) if i%2]

Generalize that with i%n so everytime that the index is divisible by n then i%n==0 and bar(line) isn't issued into the listcomp.

enumerate works for every iterable (file handle, generator ...), so it's way better than using range(len(findex))

Now the above is incorrect if you want to call bar on all the values (because you need the side effect generated by bar), because the filter prevents execution. So you have to do that in 2 passes, for instance using map to apply your function to all items of findex and pick only the results you're interested in (but it guarantees that all of the lines are processed) using the same modulo filter but after the execution:

l = [x for i,x in enumerate(map(bar,findex)) if i%n]