Tracking file load progress in Python

Question

A lot of modules I use import entire files into memory or trickle a file's contents in while they process it. I'm wondering if there's any way to track this sort of loading progress? Possibly a wrapper class that takes a callback?

Answer 1

I would do by this by determining the size of the file, and then simply dividing the total by the number of bytes read. Like this:

import os

def show_progress(file_name, chunk_size=1024):
    fh = open(file_name, "r")
    total_size = os.path.getsize(file_name)
    total_read = 0
    while True:
        chunk = fh.read(chunk_size)
        if not chunk: 
            fh.close()
            break
        total_read += len(chunk)
        print "Progress: %s percent" % (total_read/total_size)
        yield chunk

for chunk in show_progress("my_file.txt"):
    # Process the chunk
    pass 

Edit: I know it isn't the best code, but I just wanted to show the concept.