Choice of transports for JSON over TCP

Question

I'm writing a simple streaming JSON service. It consists of JSON messages, sent intermittently, for a long period of time (weeks or months).

What is the best practise with regard to sending multiple JSON messages over a plain TCP socket?

Some alternatives I have looked at (and their downsides) are:

1. newline separated JSON - downside: newlines within JSON require escaping, or prohibition
2. websocket inspired 0x00 0xff framing - downside: it's now binary, not utf-8 anymore
3. real websockets - downside: lack of (opensource) websocket client libraries
4. http multipart http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html - downside: incomplete client support?
5. no delimiters - downside: chunking requires JSON parsing (can't just count curlies because of curlies in strings)

Is there a good, or at least well-established way of doing this?

Answer 1

my first two options would be:

1. Do what early TCP protocols do: send one message (a JSON object in your case) and close the connection. The client detects it and reopens to get the next object.

   • pros: very easy to parse, no extra (content) bytes sent. any loss of data means losing just a single object. if you can stand that, there's no need to add retransmision to your app.
   • cons: if you send a (huge) lot of (very) small objects, the three-packet TCP handshake adds to latency.

2. Do what chunked-mode HTTP does: first send the number of bytes in the JSON object, a newline (CRLF in HTTP), and your JSON object. The client just have to count bytes to know when the next byte would be the next objectsize.

   • pros: you keep one long-lived stream.
   • cons: a few extra bytes, you have to keep a long-lived stream, so accidental break and reconnection has to be handled as exceptional events, need to establish some handshaking to continue where it failed.

Answer 2

When you want to serve browser clients, the closest you get to raw TCP is WebSockets.

WebSockets has sufficient momentum that browser vendors will improve support (Chrome 14 and Firefox 7/8 support the latest protocol draft) and that a broad range of client and server frameworks will support it.

There are already a couple of open-source client libraries, including Autobahn WebSocket.

When you want to bake something for your own (on top of raw TCP), I would recommend a length-prefixed format for your JSON messages, i.e. Netstrings

Disclaimer: I am author of Autobahn and work for Tavendo.