I'm trying to create a 'Download Manager' for Linux that lets me download one single file using multiple threads. This is what I'm trying to do :
Steps 2 and 3 are solvable, and it is at Step #1 that I'm stuck. How do I specify an offset while downloading a file?
Using something along the lines of open("/path/to/file", "wb").write(urllib2.urlopen(url).read())
does not let me specify a starting point to read from. Is there any alternative to this?
To download part of the file, just set the Range
header like this
req = urllib2.Request(url)
req.headers['Range'] = 'bytes=%s-%s' % (start, end)
f = urllib2.urlopen(req)
Not all server support the Range
header though. Most file sharing service don't.
first, the http server should return Content-Length header. this is usually means the file is a static file, if it is a dynamic file, such as a result of php or jsp, you can not do such split.
then, you can use http Range header when request, this header tell the server which part of file should return. see python doc for how set and parse http head.
to do this, if the part size is 100k, you first request with Range: 0-1000000 100k will get first part, and in its conent-length in response tell your the size of file, then start some thread with different Range, it will work
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With