what is wrong with json response

Question

json response which is working correctly :

obj = urllib.urlopen("http://www.omdbapi.com/?t=Fight Club")
response_str = obj.read()
response_json = simplejson.loads(response_str)

above code making the json request which looks like :

{
    "Title":"Fight Club",
    "Year":"1999",
    "Rated":"R",
    "Released":"15 Oct 1999",
     ......
    "Response":"True"
}

so i can sleep now... but

json response which is not working correctly :

obj = urllib.urlopen("https://api.stackexchange.com/2.1/answers?order=desc&sort=activity&site=stackoverflow")
response_str = obj.read()
response_json = simplejson.loads(response_str)

above code making the json request which looks like :

{

    "items": [
        {
            "question_id": 18384375,
            "answer_id": 18388044,
            "creation_date": 1377195687,
            "last_activity_date": 1377195687,
            "score": 0,
            "is_accepted": false,
            "owner": {
                "user_id": 1745001,
                "display_name": "Ed Morton",
                "reputation": 10453,
                "user_type": "registered",
                "profile_image": "https://www.gravatar.com/avatar/99a3ebae89496eb16afe453aae97f5be?s=128&d=identicon&r=PG",
                "link": "https://stackoverflow.com/users/1745001/ed-morton"
            }
        },
        {
            "question_id": 18387447,
            "answer_id": 18388040,
            "creation_date": 1377195667,
            "last_activity_date": 1377195667,
            "score": 0,
            "is_accepted": false,
            "owner": {
                "user_id": 2494429,
                "display_name": "kpark91",
                "reputation": 140,
                "user_type": "registered",
                "profile_image": "https://www.gravatar.com/avatar/d903a03e7c5b6d9b21ff598c632de575?s=128&d=identicon&r=PG",
                "link": "https://stackoverflow.com/users/2494429/kpark91"
            }
        }

       ]
}

returns

JSONDecodeError at /
No JSON object could be decoded: line 1 column 0 (char 0)

instead of using simplejson i tried json which gave following error :

ValueError at /
No JSON object could be decoded

what i tried and failed:

I tried my questions's answers on stackoverflow having same problem but none of them gave clear cut solution although every one helped me in some way

1) i checked whether json content encoding is correct or not

>>obj.info()

Content-Type: application/json; 
charset=utf-8
Content-Length: 2398

2) i decoded the response_str in utf-8

json.loads(response_str).decode("utf-8")

3) i used jsonlint to check format of json response

Parse error on line 1:

^
Expecting '{', '['

surprisingly following link of rescued my sleep. JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Basically i was using exact code to get json response in both cases , but whats wrong with the second json response, only difference i noticed that the structure of second json response was different from first one.

please provide explanation to understand the issue.

Answer 1

Let me give you some advice. Using urllib is great and all, but you will run into a lot of trouble. So, let me save you the pain, and just lead you to requests. Its the best way to deal with url requests.

This short IDLE session will demonstrate:

>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
'utf-8'
>>> r.text
u'{"type":"User"...'
>>> r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}

To install it, just go to your command prompt and pip install requets. requets is free and open source, and its written in pure python so the installation is simple.

Here is a video to demonstrate:

Your particular problem:

Answer 2

One way to have a good sleep is to use requests:

import requests
import simplejson

response = requests.get("https://api.stackexchange.com/2.1/answers?order=desc&sort=activity&site=stackoverflow")
response_json = simplejson.loads(response.text)
print response_json

No errors, just works.