performance is very bad to construct boost property tree from json file?

Question

I am using boost property tree to load/dump json file. However, the performance is very bad.

For example, I have a json file whose size is 1.8M. The boost C++ program spends 3 seconds to load the json file and construct the property tree. If I use python to load the json file, it only need 0.1 second. And python will also construct everything as object as well.

The C++ program is like:

int main(int argc, char **argv){
        std::fstream fin;
        fin.open(argv[1], std::fstream::in);
        if (!fin.is_open()){
            ASSERT(false);
        }

        boost::property_tree::ptree pt;
        try{
            read_json(fin, pt);
        }catch(ptree_error & e) {
            ASSERT(false);
        }
        fin.close();

    return 0;
}

The python script which is doing same thing is like:

#!/usr/bin//python

import sys
import json

fp = open(sys.argv[1],"r")
objs = json.load(fp)

I tried the lastest boost (1.54). It's still very slow on doing this.

Appreciate for any advice.

If there is no solution, do you know any other C++ library to load/dump json?

Answer 1

We had significant performance problems with boost::property_tree and JSON. Our approach was to stop using std::string and use an in-house string class with a custom allocator, and hash tables for not reallocating the same string twice. This improved performance and memory usage by at least a few orders of magnitude for large JSON files.

Our JSON files were large enough that the std::string allocation consumed all available address space on a 32-bit machine. This approach let us run with headroom.

Answer 2

I found that there is a huge difference between Release Build vs Debug Build performance numbers from VS for Property Tree. on my specific hardware a parsing through a 1 MB JSON File using read_json was taking 8 sec in Debug build , but only 0.7 sec in release version.