Object of type Response is not JSON serializable

Question

I am trying to convert my json from an api to a dataframe in python. I run the following codes:

import requests
import pandas as pd
import simplejson as json

params = {
  "api_key": "abc",
  "format": "json"
}
r = requests.get('https://www.parsehub.com/api/v2/runs/ttx8PT-EL6Rf/data', params=params)

data = json.dumps(r) #convert to json
data = json.loads(data)
dataFrame = pd.DataFrame.from_dict(data) #convert json to dataframe
print(dataFrame)

and I got these errors:

Traceback (most recent call last):
  File "C:/Users/User/PycharmProjects/test/convert jsontodataframe.py", line 11, in <module>
    data = json.dumps(r) #convert to json
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\simplejson\__init__.py", line 395, in dumps
    return _default_encoder.encode(obj)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\simplejson\encoder.py", line 296, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\simplejson\encoder.py", line 378, in iterencode
    return _iterencode(o, 0)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\simplejson\encoder.py", line 273, in default
    o.__class__.__name__)
TypeError: Object of type Response is not JSON serializable

can you help me please. Is there another way I can convert my json api to a dataframe in python? Please help me.

Answer 1

Remove this line:

data = json.dumps(r) #convert to json

This converts a python dictionary to a string - not a string to a json object as you have annotated in your code.

Your code should look like this:

import requests
import pandas as pd
import simplejson as json

params = {
  "api_key": "abc",
  "format": "json"
}

r = requests.get('https://www.parsehub.com/api/v2/runs/ttx8PT-EL6Rf/data', params=params)

data = json.loads(r.json)
dataFrame = pd.DataFrame.from_dict(data) #convert json to dataframe
print(dataFrame)

Answer 2

r.json() is an already decoded Python object. Here's an example from a site without an API key:

import requests
import pandas as pd
import json

params = {"s": "margarita"}

r = requests.get('https://www.thecocktaildb.com/api/json/v1/1/search.php', params=params)

data = r.json()
dataFrame = pd.DataFrame.from_dict(data['drinks'])
print(dataFrame)

Output:

  idDrink                      strDrink  ... strCreativeCommonsConfirmed         dateModified
0   11007                     Margarita  ...                         Yes  2015-08-18 14:42:59
1   11118                Blue Margarita  ...                         Yes  2015-08-18 14:51:53
2   17216             Tommy's Margarita  ...                          No  2017-09-02 18:37:54
3   16158            Whitecap Margarita  ...                          No  2015-09-02 17:00:22
4   12322          Strawberry Margarita  ...                          No  2015-08-18 14:41:51
5  178332  Smashed Watermelon Margarita  ...                          No                 None

[6 rows x 51 columns]