Parsing HTTP Response in Python

Question

I want to manipulate the information at THIS url. I can successfully open it and read its contents. But what I really want to do is throw out all the stuff I don't want, and to manipulate the stuff I want to keep.

Is there a way to convert the string into a dict so I can iterate over it? Or do I just have to parse it as is (str type)?

from urllib.request import urlopen  url = 'http://www.quandl.com/api/v1/datasets/FRED/GDP.json' response = urlopen(url)  print(response.read()) # returns string with info 

Answer 1

When I printed response.read() I noticed that b was preprended to the string (e.g. b'{"a":1,..). The "b" stands for bytes and serves as a declaration for the type of the object you're handling. Since, I knew that a string could be converted to a dict by using json.loads('string'), I just had to convert the byte type to a string type. I did this by decoding the response to utf-8 decode('utf-8'). Once it was in a string type my problem was solved and I was easily able to iterate over the dict.

I don't know if this is the fastest or most 'pythonic' way of writing this but it works and theres always time later of optimization and improvement! Full code for my solution:

from urllib.request import urlopen import json  # Get the dataset url = 'http://www.quandl.com/api/v1/datasets/FRED/GDP.json' response = urlopen(url)  # Convert bytes to string type and string type to dict string = response.read().decode('utf-8') json_obj = json.loads(string)  print(json_obj['source_name']) # prints the string with 'source_name' key