Sending/Parsing multiple JSON objects

Question

I have a Sinatra server that is returning multiple JSON objects from the database in a streaming manner. The objects would look like:

{"a": 1, "b": 2, "c": 3}
{"a": 4, "b": 5, "c": 6}
...

but this is invalid JSON. I can add a hack into Sinatra's event processing (manually injecting the missing array delimiters) to make the response look like:

[
{"a": 1, "b": 2, "c": 3}
, {"a": 4, "b": 5, "c": 6}
]

which is valid JSON now, but this technique is inelegant. Is there some way to do this client-side? Basically, what I want is to have a JavaScript function read a string and consume a valid JSON object, and then return to me the JSON object and the remainder of the string, iteratively being called until the entire string is consumed.

Answer 1

The native JSON.parse() function expect the whole string to be valid JSON. I'm not aware of a parser that only consumes the first valid object as you'd like. And people should really be producing valid JSON anyways.

If you know that there is one object per line you could simply split the string by line using the split() function and parse each line individually.

var str = '{"a": 1, "b": 2, "c": 3}\n'+
          '{"a": 4, "b": 5, "c": 6}';

var strLines = str.split("\n");


for (var i in strLines) {
  var obj = JSON.parse(strLines[i]);
  console.log(obj.a);
}

You could also use a bit of string manipulation to transform each line into an array element and parse the whole thing.

str = "["+str.replace(/\n/g, ",")+"]";
JSON.parse(str);

Answer 2

I would do this:

var str = '{"a": 1, "b": 2, "c": 3}{"a": 4, "b": 5, "c": 6}';

var res = JSON.parse('[' + str.replace(/}{/g, '},{') + ']');

Edit:

as awnser on tremby's comment

var str = '{"a": 1, "b": 2, "c": 3}{"a": 4, "b": 5, "c": 6}';

var res = JSON.parse('[' + str.replace(/}{(?=([^"]*"[^"]*")*[^"]*$)/g, '},{') + ']');

Answer 3

I wrote a small module today to do this and published it on NPM as json-multi-parse. The code is available on Github.

My solution is simple, but admittedly possibly brittle since it relies on the error message JSON.parse throws when parsing such a string. It uses the position number given in the error (the number in "Unexpected token { in JSON at position xyz") to parse everything up to before that, then recurse and parse everything after.

However, it won't break due to curly braces in strings as some of the other suggestion solutions here will.

Here's the simple version of the code, which will work in Chrome and Node.

const ERROR_REGEX = /^Unexpected token { in JSON at position (\d+)$/;
function jsonMultiParse(input, acc = []) {
    if (input.trim().length === 0) {
        return acc;
    }
    try {
        acc.push(JSON.parse(input));
        return acc;
    } catch (error) {
        const match = error.message.match(ERROR_REGEX);
        if (!match) {
            throw error;
        }
        const index = parseInt(match[1], 10);
        acc.push(JSON.parse(input.substr(0, index)));
        return jsonMultiParse(input.substr(index), acc);
    }
}

It gets more complicated if you want to support Firefox too, which gives its error in a format giving line number and character within that line. The module I linked above handles this case.