Python urllib won't download file due to permissions, but wget will

Question

I'm trying to download an MP3 file, via its URL, using Python's urllib2.

mp3file = urllib2.urlopen(url)
output = open(dst,'wb')
output.write(mp3file.read())
output.close()

I'm getting a urllib2.HTTPError: HTTP Error 403: Forbidden error. Trying urllib also fails, but silently.

urllib.urlretrieve(url, dst)

However, if I use wget, I can download the file successfully.

I've noted the general differences between the two methods mentioned in "Difference between Python urllib.urlretrieve() and wget", but they don't seem to apply here.

Is wget doing something to negotiate permissions that urllib2 doesn't do? If so, what, and how do I replicate this in urllib2?

Answer 1

Could be something on the server side - blocking python user agent for example. Try using wget user agent : Wget/1.13.4 (linux-gnu) .

In Python 2:

import urllib

# Change header for User-Agent
class AppURLopener(urllib.FancyURLopener):
    version = "Wget/1.13.4 (linux-gnu)"
url = "http://www.example.com/test_file"
fname = "test_file"
urllib._urlopener = AppURLopener()
urllib.urlretrieve(url, fname)