I would like to use itertools.islice(self._f, 0, 100, None) to read in a file piece by piece (in blocks of 100 lines) as follows:
f = open('test.dat', 'r')
while (some condition I look for):
f = open(fileName, 'r')
x = itertools.islice(f, 0, 100, None)
doSomethingWithX(x)
My problem is, I do not know how long the file is and I am looking for a condition to stop the while loop when the end of the file is reached. But I cannot figure out how it is done.
EDIT: Ok, I see the difficulty. Maybe I should reformulate the question when the itertools.islice is capsuled in a class like here:
class reader:
def __init__()
self._f = open('test.dat', 'r')
def getNext():
return itertools.islice(self._f, 0, 100, None)
R = reader()
while (some condition I look for):
x = R.getNext()
doSomethingWithX(x)
If you don't mind getting list slices, you can use iter:
with open(filename, 'r') as f:
for x in iter(lambda: list(itertools.islice(f, 100)), []):
doSomethingWithX(x)
Not sure which file you are using as you have f = .. twice and have self_.f in there too.
Using your edited code:
class reader:
def __init__(self):
self._f = open('out.csv', 'r')
def getNext(self):
return itertools.islice(self._f, 100)
R = reader()
import itertools
for x in iter(lambda: list(R.getNext()),[]):
print(x)
using a test file with the following and your class code using itertools.islice(self._f, 2):
1
2
3
4
5
6
7
8
9
10
outputs:
In [15]: R = reader()
In [16]: import itertools
In [17]: for x in iter(lambda: list(R.getNext()),[]):
....: print(x)
....:
['1\r\n', '2\r\n']
['3\r\n', '4\r\n']
['5\r\n', '6\r\n']
['7\r\n', '8\r\n']
['9\r\n', '10']
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With