I'm using
data=urllib2.urlopen(url).read()
I want to know:
How can I tell if the data at a URL is gzipped?
Does urllib2 automatically uncompress the data if it is gzipped? Will the data always be a string?
- How can I tell if the data at a URL is gzipped?
This checks if the content is gzipped and decompresses it:
from StringIO import StringIO import gzip request = urllib2.Request('http://example.com/') request.add_header('Accept-encoding', 'gzip') response = urllib2.urlopen(request) if response.info().get('Content-Encoding') == 'gzip': buf = StringIO(response.read()) f = gzip.GzipFile(fileobj=buf) data = f.read()
- Does urllib2 automatically uncompress the data if it is gzipped? Will the data always be a string?
No. The urllib2 doesn't automatically uncompress the data because the 'Accept-Encoding' header is not set by the urllib2 but by you using: request.add_header('Accept-Encoding','gzip, deflate')
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With