Use "byte-like object" from urlopen.read with JSON?

Question

Just trying to test out very simple Python JSON commands, but I'm having some trouble.

urlopen('http://www.similarsitesearch.com/api/similar/ebay.com').read()

should output

'{"num":20,"status":"ok","r0":"http:\\/\\/www.propertyroom.com\\/","r1":"http:\\/\\/www.ubid.com\\/","r2":"http:\\/\\/www.bidcactus.com\\/","r3":"http:\\/\\/www.etsy.com\\/","r4":"http:\\/\\/us.ebid.net\\/","r5":"http:\\/\\/www.bidrivals.com\\/","r6":"http:\\/\\/www.ioffer.com\\/","r7":"http:\\/\\/www.shopgoodwill.com\\/","r8":"http:\\/\\/www.beezid.com\\/","r9":"http:\\/\\/www.webidz.com\\/","r10":"http:\\/\\/www.auctionzip.com\\/","r11":"http:\\/\\/www.overstock.com\\/","r12":"http:\\/\\/www.bidspotter.com\\/","r13":"http:\\/\\/www.paypal.com\\/","r14":"http:\\/\\/www.ha.com\\/","r15":"http:\\/\\/www.onlineauction.com\\/","r16":"http:\\/\\/bidz.com\\/","r17":"http:\\/\\/www.epier.com\\/","r18":"http:\\/\\/www.sell.com\\/","r19":"http:\\/\\/www.rasmus.com\\/"}'

but I get that same string, with a b in front:

b'{"num":20,"status":"ok","r0":"http:\\/\\/www.propertyroom.com\\/","r1":"http:\\/\\/www.ubid.com\\/","r2":"http:\\/\\/www.bidcactus.com\\/","r3":"http:\\/\\/www.etsy.com\\/","r4":"http:\\/\\/us.ebid.net\\/","r5":"http:\\/\\/www.bidrivals.com\\/","r6":"http:\\/\\/www.ioffer.com\\/","r7":"http:\\/\\/www.shopgoodwill.com\\/","r8":"http:\\/\\/www.beezid.com\\/","r9":"http:\\/\\/www.webidz.com\\/","r10":"http:\\/\\/www.auctionzip.com\\/","r11":"http:\\/\\/www.overstock.com\\/","r12":"http:\\/\\/www.bidspotter.com\\/","r13":"http:\\/\\/www.paypal.com\\/","r14":"http:\\/\\/www.ha.com\\/","r15":"http:\\/\\/www.onlineauction.com\\/","r16":"http:\\/\\/bidz.com\\/","r17":"http:\\/\\/www.epier.com\\/","r18":"http:\\/\\/www.sell.com\\/","r19":"http:\\/\\/www.rasmus.com\\/"}'

Subsequently, when I try to run

json.loads(urlopen('http://similarsitesearch.com/api/similar/ebay.com').read())

it gives me the error message:

TypeError: can't use a string pattern on a bytes-like object"

which I'm assuming has something to do with the b?

I imported urlopen from urllib.request, and I am running Python 3.

Any ideas?

Answer 1

The content from read() is of type bytes so you need to convert it to a string before trying to decode it into a json object.

To convert bytes to a string, change your code to: urlopen('http://similarsitesearch.com/api/similar/ebay.com').read().decode("utf-8")

Answer 2

You need to examine the charset specified in the Content-Type header and decode by that before passing it to json.load*().