Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parse ~1 MB JSON on Android very slow

I have an approximately 1MB JSON file stored in my assets folder that I need to load in my app every time it runs. I find that the built-in JSON parser (org.json) parses the file very slowly, however once it's parsed, I can access and manipulate the data very quickly. I've counted out as many as 7 or 8 seconds from the moment I click on the app to the moment the Activity1 is brought up, but just a few milliseconds to go from Activity1 to Activity2 which depends on data processed from the data loaded in Activity1.

I'm reading the file into memory and parsing it using:

String jsonString = readFileToMemory(myFilename)
JSONArray array = new JSONArray(jsonString);

where readFileToMemory(String) looks like this:

private String readFileToMemory(String filename) {
    StringBuilder data = new StringBuilder();       
    BufferedReader reader = null;
    try {
        InputStream stream = myContext.getAssets().open(filename);
        reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
        int result = 0;
        do
        {
            char[] chunk = new char[512];
            result = reader.read(chunk);
            data.append(chunk);
        }
        while(result != -1);

    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
    return data.toString();
}

Does anyone have any suggestions to how I can speed up the initial loading and parsing of the data? Should I perhaps mask the whole process behind a loading screen?

like image 833
rdavison Avatar asked Aug 29 '12 03:08

rdavison


People also ask

Is parsing JSON slow?

test with JSON. parse: 702 ms. Clearly JSON. parse is the slowest of them, and by some margin.

How long does it take to parse JSON?

In any case, to answer my own question, it seems that parsing JSON should take about 8 cycles per input byte on a recent Intel processor. Maybe less if you are clever. So you should expect to spend 2 or 3 seconds parsing one gigabyte of JSON data.

What is the fastest JSON parser?

We released simdjson 0.3: the fastest JSON parser in the world is even better! Last year (2019), we released the simjson library. It is a C++ library available under a liberal license (Apache) that can parse JSON documents very fast.

Is JSON difficult to parse?

Actually unless one is doing JavaScript, JSON is extremely difficult to parse correctly.


2 Answers

JSONObject -- the one from json.org, is the simplest API to use to parse JSON. However it comes with a cost -- performace. I have done extensive experiments with JSONObject, Gson and Jackson. It seems no matter what you do, JSONObject (and hence JSONArray) will be the slowest. Please switch to Jackson or Gson.

Here is the relative performance of the two

(fastest) Jackson > Gson >> JSONObject (slowest)

Refer:
- Jackson
- Gson

like image 164
Nishant Avatar answered Oct 17 '22 19:10

Nishant


You should make an SQLite table to store the data and move it from JSON to SQL the first time the app runs. As an added benefit, this makes the data easier to search through and makes it possible for you to modify the data from within the app.

like image 38
mimicocotopus Avatar answered Oct 17 '22 18:10

mimicocotopus