Making JSON responses even smaller... just an idea

Question

Just a thought really... and wondering if Gzipped JSON already covers this.

But say you have a list of game objects in a response:

game = {
    name: 'Randomer Quest!',
    description: 'Randomer's Quest is a brilliant game!',
    activated: true,
    points: 10,
    thumb: 'randomer-quest.jpg'
};

When you json_encode this, it becomes 151 bytes:

{"games": [{"name":"Randomer Quest!","description":"Randomer's Quest is a brilliant game!","activated":true,"points":10,"thumb":"randomer-quest.jpg"}]}

Ok... but what if you have a list of about 100 games? That's about 13,913 bytes... but do we really need to keep declaring those properties? I know you can just decode it and loop through it (the magic) but what if we're a little more intelligent about it and declare the properties in a seperate object and then have an array of data? We'd have to prefill properties that aren't there usually but I still think its worth it.

Something like this:

{
"games": {
    p: ["name", "description", "activated", "points", "thumb"],
    d: [
        ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"],
        ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"]
    ]
}

}

P are properties, D is the data in arrays. Afterwards we have: 9,377 bytes 67% of the size!

Ok I know you're going to say that's nothing but you do see requests that are more like 40-100kb. And I think that's quite a massive difference. Anyone employing something like this already? Perhaps we have tools that already do this automatically?

32bitkid has pretty much said that if you were going to do this, you might as well just trim it down to CSV format... which makes sense... that would be around 9,253 bytes 66.5%.

"name", "description", "activated", "points", "thumb"
"Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"
"Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"

I've seen JSON requests of about 100kb, which would turn into 66.5kb (losing 33.5kb)

What do you think?

Dom

Answer 1

I agree this is much more compact.

{
    "games": {
        p: ["name", "description", "activated", "points", "thumb"],
        d: [
            ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"],
            ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"]
        ]
    }
}

But wait, you could optimize it further, do you really need the "games" object? this is even smaller!

{
    p: ["name", "description", "activated", "points", "thumb"],
    d: [
        ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"],
        ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"]
    ]
}

And really, whats the point of the "p" and "d" and the object that contains, i know that the property names are going to be first, and my data is going to be second?

[
    ["name", "description", "activated", "points", "thumb"],
    ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"],
    ["Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"]
]

And those array and object markers are just getting in the way, save a few more bytes!

"name", "description", "activated", "points", "thumb"
"Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"
"Randomer Quest!", "Randomer's Quest is a brilliant game!", true, 10, "randomer-quest.jpg"

Wait... this format already exists. It is CSV. And its been around since the mid 1960's. And its part of the reason why XML and JSON were invented in the first place. JSON and XML add flexibility to the objects being stored and to make them more human readable than tightly packed binary objects. Are you really that worried about the size of the data going over the pipe? If you are (if that is, in fact, your bottleneck) then there are a bunch of different ways to address that problem.

But, personally, I think you should use the technology and the tools for what they are made for, and what they excel at doing.

You're trying to use a hammer to screw in a screw... You'll get it in the wall, but it wont be pretty or pleasant for either party involved.

Find a pattern that solves your problem, not the other way around.