Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python urllib2 Progress Hook

I am trying to create a download progress bar in python using the urllib2 http client. I've looked through the API (and on google) and it seems that urllib2 does not allow you to register progress hooks. However the older deprecated urllib does have this functionality.

Does anyone know how to create a progress bar or reporting hook using urllib2? Or are there some other hacks to get similar functionality?

like image 679
speedplane Avatar asked Jan 08 '10 15:01

speedplane


1 Answers

Here's a fully working example that builds on Anurag's approach of chunking in a response. My version allows you to set the the chunk size, and attach an arbitrary reporting function:

import urllib2, sys  def chunk_report(bytes_so_far, chunk_size, total_size):    percent = float(bytes_so_far) / total_size    percent = round(percent*100, 2)    sys.stdout.write("Downloaded %d of %d bytes (%0.2f%%)\r" %         (bytes_so_far, total_size, percent))     if bytes_so_far >= total_size:       sys.stdout.write('\n')  def chunk_read(response, chunk_size=8192, report_hook=None):    total_size = response.info().getheader('Content-Length').strip()    total_size = int(total_size)    bytes_so_far = 0     while 1:       chunk = response.read(chunk_size)       bytes_so_far += len(chunk)        if not chunk:          break        if report_hook:          report_hook(bytes_so_far, chunk_size, total_size)     return bytes_so_far  if __name__ == '__main__':    response = urllib2.urlopen('http://www.ebay.com');    chunk_read(response, report_hook=chunk_report) 
like image 193
Kenan Banks Avatar answered Sep 17 '22 00:09

Kenan Banks