Use Jackson To Stream Parse an Array of Json Objects

Question

I have a file that contains a json array of objects:

[ { "test1": "abc" }, { "test2": [1, 2, 3] } ]

I wish to use use Jackson's JsonParser to take an inputstream from this file, and at every call to .next(), I want it to return an object from the array until it runs out of objects or fails.

Is this possible?

Use case: I have a large file with a json array filled with a large number of objects with varying schemas. I want to get one object at a time to avoid loading everything into memory.

EDIT:

I completely forgot to mention. My input is a string that is added to over time. It slowly accumulates json over time. I was hoping to be able to parse it object by object removing the parsed object from the string.

But I suppose that doesn't matter! I can do this manually so long as the jsonParser will return the index into the string.

Answer 1

Yes, you can achieve this sort of part-streaming-part-tree-model processing style using an ObjectMapper:

ObjectMapper mapper = new ObjectMapper();
JsonParser parser = mapper.getFactory().createParser(new File(...));
if(parser.nextToken() != JsonToken.START_ARRAY) {
  throw new IllegalStateException("Expected an array");
}
while(parser.nextToken() == JsonToken.START_OBJECT) {
  // read everything from this START_OBJECT to the matching END_OBJECT
  // and return it as a tree model ObjectNode
  ObjectNode node = mapper.readTree(parser);

  // do whatever you need to do with this object
}

parser.close();

Answer 2

What you are looking for is called Jackson Streaming API. Here is a code snippet using Jackson Streaming API that could help you to achieve what you need.

JsonFactory factory = new JsonFactory();
JsonParser parser = factory.createJsonParser(new File(yourPathToFile));

JsonToken token = parser.nextToken();
if (token == null) {
    // return or throw exception
}

// the first token is supposed to be the start of array '['
if (!JsonToken.START_ARRAY.equals(token)) {
    // return or throw exception
}

// iterate through the content of the array
while (true) {

    token = parser.nextToken();
    if (!JsonToken.START_OBJECT.equals(token)) {
        break;
    }
    if (token == null) {
        break;
    }

    // parse your objects by means of parser.getXxxValue() and/or other parser's methods

}