Python JSON decode ValueError: Extra data:

Question

I'm having a problem trying to decode and print JSON I receive from a socket connection.

The full traceback:

C:\Users\Jeremy>python F:\Files\Python\test.py
2013-01-04 21:15:35 [INFO] [AutoSaveWorld] World save Complete!
2013-01-04 21:15:50 [INFO] [←[34;1mMain←[37;1m]←[32;22mRexOZ←[37;1m: you cahaned
 your house it looks awesome←[m
Traceback (most recent call last):
  File "F:\Files\Its safer indoors\Python\test.py", line 14, in <module>
    data = json.loads(dreceve)
  File "C:\Python33\lib\json\__init__.py", line 309, in loads
    return _default_decoder.decode(s)
  File "C:\Python33\lib\json\decoder.py", line 355, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 2 column 1 - line 3 column 1 (char 151 - 344)

As you can see the first 2 lines print fine and then it crashes.

Full code:

import socket
import json
import re

HOST = 'host.host.net'
PORT = 8082
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
dsend = "/api/subscribe?source=console&key=SUPERSEXYSECRETEY&show_previous=true\n"
s.sendall(dsend.encode())

while 1:
    dreceve = s.recv(1024).decode()
    data = json.loads(dreceve)
    succses = data['success']
    line = succses['line']
    print(line)

s.close()

I have goggled around for this error and the pages I found did not solve my problem, any help work be appreciated.

Answer 1

Whatever you receive, it does not seem to end where it should end; example:

>>> import json
>>> json.loads(""" {"Hello" : "World"} \ """)
....
ValueError: Extra data: line 1 column 21 - line 1 column 23 (char 21 - 23)

I'd suggest inspect your output before it gets parsed to get hold of the problem.

PS. There are simpler ways to get JSON data from a server (assuming your server returns parsable JSON, which it might not). Here is an example using the requests library:

>>> import json, requests
>>> u = "http://gdata.youtube.com/feeds/api/standardfeeds/most_popular?alt=json"
>>> json.loads(requests.get(u).text) # <-- request + parse
{u'feed': {u'category': [{u'term': u'http://gdata.youtube.com/...

.....

Answer 2

In the json lib ->decoder.py->decode function

if end != len(s):
       raise ValueError(errmsg("Extra data" , s , end , len(s)))

It's mean if your string's len != end , will raise this Exception And the end is the last "}" postion in your string. so you I can use:

string = "".join([string.rsplit("}" , 1)[0] , "}"]) 

cut the extra data after the last "}".