HTTPResponse' object has no attribute 'decode

Question

I was getting the following error initially when I was trying to run the code below-

Error:-the JSON object must be str, not 'bytes' 

import urllib.request
import json
search = '230 boulder lane cottonwood az'
search = search.replace(' ','%20')
places_api_key = 'AIzaSyDou2Q9Doq2q2RWJWncglCIt0kwZ0jcR5c'
url = 'https://maps.googleapis.com/maps/api/place/textsearch/json?query='+search+'&key='+places_api_key
json_obj = urllib.request.urlopen(url)
data = json.load(json_obj)
for item in  data ['results']:
     print(item['formatted_address'])
     print(item['types'])

After making some troubleshooting changes like:-

 json_obj = urllib.request.urlopen(url)
 obj = json.load(json_obj)
 data = json_obj .readall().decode('utf-8')

 Error - 'HTTPResponse' object has no attribute 'decode'

I am getting the error above, I have tried multiple posts on stackoverflow nothing seem to work. I have uploaded the entire working code if anyone can get it to work I'll be very grateful. What I don't understand is that why the same thing worked for others and not me. Thanks!

Answer 1

urllib.request.urlopen returns an HTTPResponse object which cannot be directly json decoded (because it is a bytestream)

So you'll instead want:

# Convert from bytes to text
resp_text = urllib.request.urlopen(url).read().decode('UTF-8')
# Use loads to decode from text
json_obj = json.loads(resp_text)

However, if you print resp_text from your example, you'll notice it is actually xml, so you'll want an xml reader:

resp_text = urllib.request.urlopen(url).read().decode('UTF-8')
(Pdb) print(resp_text)
<?xml version="1.0" encoding="UTF-8"?>
<PlaceSearchResponse>
  <status>OK</status>
...

update (python3.6+)

In python3.6+, json.load can take a byte stream (and json.loads can take a byte string)

This is now valid:

json_obj = json.load(urllib.request.urlopen(url))