Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python read 1000 at a time until end of file?

Tags:

python

file-io

I've been working on a python project to send information to a server, and while I found some nice code to read the file and send it, I can only send the entire thing at once. I want instead to be able to read 1000 characters at a time, and loop until the file's end. I'm sure that this is pretty easy, but I'm not sure where to change what I've been doing to make it happen. This might also be due to unfamiliarity with the method of reading, but I am researching it and not finding any information that has clarified it for me. This is what I'm working with:

with open(localFileTT, 'rb', 0) as f: #open the file 
    for fstr in f:
        str.encode(fstr)
        print(fstr)
        if 'str' in fstr:
            break

I have not yet introduced network functions in this code, as it is only test material to figure out how to read only 1000 at a time. It certainly reads the entire section well enough, but I don't know how to change it so as to read only parts at a time. I would try just adding read(1000) but that prints emptiness! I'm not sure where exactly the reading is occurring here, and unfortunately the articles I've read online thus far on parts of this code (such as with) has not helped any. If anyone can point me in the right direction, then I will be quite grateful.

like image 229
user2828965 Avatar asked Jan 12 '23 01:01

user2828965


1 Answers

file.read() takes a size argument:

Read at most size bytes from the file (less if the read hits EOF before obtaining size bytes). If the size argument is negative or omitted, read all data until EOF is reached.

You can use that in a loop, combined with the iter() function (with the second argument set to '' as a sentinel), and functools.partial() for efficiency:

from functools import partial

with open(localFileTT, 'rb', 0) as f:
    for chunk in iter(partial(f.read, 1000), ''):
        # chunk is up to 1000 characters long

The alternative would be to use a while loop:

with open(localFileTT, 'rb', 0) as f:
    while True:
        chunk = f.read(1000)
        if not chunk:
            # EOF reached, end loop
            break
        # chunk is up to 1000 characters long

If you already tried read(1000) and got an empty string back, your file was already at EOF at the time.

like image 99
Martijn Pieters Avatar answered Jan 21 '23 23:01

Martijn Pieters