Saving and Loading FlannBasedMatcher

Question

Training an OpenCV DescriptorMatcher can be a time consuming operation if the training image set is large. So it seems that saving the trained DescriptorMatcher data to disk for reloading later would be a pretty obvious thing to want to do.

Unfortunately there does not appear to be any obvious solution to this need.

The closest I've come to finding an answer is this thread in the OpenCV discussion group. The thread started in 2009, and people are still looking for an answer in 2011!

This code snippet taken from that thread looks like it should reload an index from a file:

FileStorage fs("data.xml",FileStorage::READ);
Mat data;
fs["mtx"] >> data;
flann::Index idx(data,"index.bin");

But I haven't been able to figure out from this how to implement complete save/load functionality.

Just in case it is relevant, I'm using OpenCV 2.3.1.

Answer 1

I did not see an answer to this on forums or mailing list. I had to dig into the OpenCV source code (2.4.5) to see how this needs to be done. It requires subclassing to get at the protected members of FlannBasedMatcher.

The key is setting the algorithm to FLANN_INDEX_SAVED and the file name on the indexParams.

Also of note:

• The descriptors must be passed to add() before readIndex()

• For the index to be built, you must do a match on it first, then call write(). train() does not seem to do anything except construct the matcher (doesn't feed it descriptors)

• This works with SURF descriptors. For a complete solution it may be necessary to save/restore the IndexParams and/or SearchParams of the matcher as well.

Next thing to do is compress the index (with gzip), it can be 3-4 times smaller and cost to decompress is relatively low. This would have to be a patch in OpenCV.

class SaveableMatcher : public cv::FlannBasedMatcher
{
public:
SaveableMatcher()
{
}

virtual ~SaveableMatcher()
{
}

void printParams()
{
    printf("SaveableMatcher::printParams: \n\t"
        "addedDescCount=%d\n\t"
        "flan distance_t=%d\n\t"
        "flan algorithm_t=%d\n",
        addedDescCount,
        flannIndex->getDistance(),
        flannIndex->getAlgorithm());

    vector<std::string> names;
    vector<int> types;
    vector<std::string> strValues;
    vector<double> numValues;

    indexParams->getAll(names, types, strValues, numValues);

    for (size_t i = 0; i < names.size(); i++)
        printf("\tindex param: %s:\t type=%d val=%s %.2f\n",
                names[i].c_str(), types[i],
                strValues[i].c_str(), numValues[i]);

    names.clear();
    types.clear();
    strValues.clear();
    numValues.clear();
    searchParams->getAll(names, types, strValues, numValues);

    for (size_t i = 0; i < names.size(); i++)
        printf("\tsearch param: %s:\t type=%d val=%s %.2f\n",
                names[i].c_str(), types[i],
                strValues[i].c_str(), numValues[i]);
}

void readIndex(const char* filename)
{
    indexParams->setAlgorithm(cvflann::FLANN_INDEX_SAVED);
    indexParams->setString("filename", filename);

    // construct flannIndex now, so printParams works
    train();

    printParams();
}

void writeIndex(const char* filename)
{
    printParams();
    flannIndex->save(filename);
}
};

Answer 2

In OpenCV 2.4.0 (but also in 2.3.1a) there are:

// Reads matcher object from a file node
virtual void read( const FileNode& );
// Writes matcher object to a file storage
virtual void write( FileStorage& ) const;

that are implemented at least for the FlannDescriptorMatcher but the implementation seems to save just the IndexParams of the matcher. Instead flann::Index_ has a save and load methods (in 2.3.1 there's save while load seems to be available using a SavedIndexParams