Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it worth the effort to try to reduce JSON size?

I am submitting relatively lots of data from a mobile application (up to 1000 JSON objects), that I would normally encode like this:

[{     id: 12,     score: 34,     interval: 5678,     sub: 9012 }, {     id: ... }, ...] 

I could make the payload smaller by submitting an array of arrays instead:

[[12, 34, 5678, 9012], [...], ...] 

to save some space on the property names, and recreate the objects on the server (as the schema is fixed, or at least it is a contract between the server and the client).

The payload in then submitted in a POST request, most likely over a 3G connection (or could be wifi).

It looks like I am saving some bandwidth by using nested arrays, but I'm not sure it is noticeable when gzip is applied, and I'm not sure how to precisely and objectively measure the difference.

On the other hand, the nested arrays don't feel like a good idea: they are less readable and thus harder to spot errors while debugging. Also, since we're flushing readability down the toilet, we could just flatten the array, since each child array has a fixed number of elements, the server could just slice it up and reconstruct the objects again.

Any further reading material on this topic is much appreciated.

like image 544
Attila O. Avatar asked Jun 22 '12 17:06

Attila O.


People also ask

How big is too big for a JSON file?

How large can JSON Documents be? One of the more frequently asked questions about the native JSON data type, is what size can a JSON document be. The short answer is that the maximum size is 1GB.

How well does JSON compress?

So even without compression, we remove nearly two-thirds of the original data. And compression shrinks our JSON data to just one-seventh of the original size!

How many lines of JSON is too much?

The maximum length of JSON strings. The default is 2097152 characters, which is equivalent to 4 MB of Unicode string data. JSON itself has no inherent limit.


2 Answers

JSONH, aka hpack, https://github.com/WebReflection/JSONH does something very similar to your example:

[{     id: 12,     score: 34,     interval: 5678,     sub: 9012 }, {     id: 98,     score: 76,     interval: 5432,     sub: 1098 }, ...] 

Would turn into:

[["id","score","interval","sub"],12,34,5678,9012,98,76,5432,1098,...] 
like image 81
John Gietzen Avatar answered Oct 06 '22 08:10

John Gietzen


JSON is meant for readability. You could have an intermediate format if you're concerned about space. Create a serialize/deserialize function which takes a JSON file and creates a compressed binary storing your data as compactly as is reasonable, then read that format on the other end of the line.

See: http://en.wikipedia.org/wiki/Json First sentence: "JSON...is a lightweight text-based open standard designed for human-readable data interchange."

Essentially, my point is that humans would always see the JSON, and machines would primarily see the binary. You get the best of both worlds: readability and small data transfer (at the cost of a tiny amount of computation).

like image 40
artoonie Avatar answered Oct 06 '22 10:10

artoonie