Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Read Multiple JSON object from a Text File

My Question is similar to what has been asked here . few points :

  1. I can not change the format. (No commas to be added etc)
  2. This is basically a huge .txt file containing 1000's of Json objects.
  3. My Json objects are HUGE.

This is what I am doing right now :

    FileReader fileReader = new FileReader(fileName);
        BufferedReader reader = new BufferedReader(fileReader);
        String data = "";
        while((data = reader.readLine()) != null){
            ObjectMapper mapper = new ObjectMapper();
            Map<String,String> map = mapper.readValue(data, Map.class);
        }

Currently I am using Jackson and Ideally I would like to read one Json Object from the file at a time, Parse it and then move on to the next one. I need to count let say unique number of id's from these Json object and do more operations. It will be best to read them one by one.

Is jackson would be the best way going forward ? This is a good example of parsing huge Json, But it deals with only one object per file. My file has huge Jsons (1000s of them).

like image 245
user1324887 Avatar asked Aug 07 '15 06:08

user1324887


1 Answers

Here is a Jackson example that works for me. I have thousands json objects (tokens) in a single json file. This code will iterate through the file read each token and print it's serial.

Required imports:

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.ObjectMapper;

Using Jackson to read multiple json objects from FileInputStream:

try (FileInputStream fis = new FileInputStream("D:/temp/tokens.json")) {
        JsonFactory jf = new JsonFactory();
        JsonParser jp = jf.createParser(fis);
        jp.setCodec(new ObjectMapper());
        jp.nextToken();
        while (jp.hasCurrentToken()) {
            Token token = jp.readValueAs(Token.class);
            jp.nextToken();
            System.out.println("Token serial "+token.getSerialNumber());
        }
    }
like image 64
Dusan.czh Avatar answered Oct 10 '22 05:10

Dusan.czh