Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to read JSON from socket in python? (Incremental parsing of JSON)

I have a socket opened and I'd like to read some json data from it. The problem is that the json module from standard library can only parse from strings (load only reads the whole file and calls loads inside) It even looks that all the way inside the module it all depends on the parameter being string.

This is a real problem with sockets since you can never read it all to string and you don't know how many bytes to read before you actually parse it.

So my questions are: Is there a (simple and elegant) workaround? Is there another json library that can parse data incrementally? Is it worth writing it myself?

Edit: It is XBMC jsonrpc api. There are no message envelopes, and I have no control over the format. Each message may be on a single line or on several lines. I could write some simple parser that needs only getc function in some form and feed it using s.recv(1), but this doesn't as a very pythonic solution and I'm a little lazy to do that :-)

like image 390
cube Avatar asked Sep 07 '11 16:09

cube


People also ask

How do you parse a JSON file in Python?

If you need to parse a JSON string that returns a dictionary, then you can use the json. loads() method. If you need to parse a JSON file that returns a dictionary, then you can use the json. load() method.

How do you parse nested JSON data in Python?

Start by importing the json library. We use the function open to read the JSON file and then the method json. load() to parse the JSON string into a Python dictionary called superHeroSquad. That's it!

What is JSON parse () method?

The JSON.parse() method parses a JSON string, constructing the JavaScript value or object described by the string. An optional reviver function can be provided to perform a transformation on the resulting object before it is returned.


1 Answers

If you're getting the JSON from an HTTP stream, use the Content-Length header to get the length of the JSON data. For example:

import httplib
import json

h = httplib.HTTPConnection('graph.facebook.com')
h.request('GET', '/19292868552')
response = h.getresponse()
content_length = int(response.getheader('Content-Length','0'))

# Read data until we've read Content-Length bytes or the socket is closed
data = ''
while len(data) < content_length or content_length == 0:
    s = response.read(content_length - len(data))
    if not s:
        break
    data += s

# We now have the full data -- decode it
j = json.loads(data)
print j
like image 110
Adam Rosenfield Avatar answered Oct 15 '22 21:10

Adam Rosenfield