Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

C++ map performance - Linux (30 sec) vs Windows (30 mins) !

I need to process a list of files. The processing action should not be repeated for the same file. The code I am using for this is -

using namespace std;

vector<File*> gInputFileList; //Can contain duplicates, File has member sFilename
map<string, File*> gProcessedFileList; //Using map to avoid linear search costs

void processFile(File* pFile)
{
    File* pProcessedFile = gProcessedFileList[pFile->sFilename];
    if(pProcessedFile != NULL)
        return; //Already processed

    foo(pFile); //foo() is the action to do for each file
    gProcessedFileList[pFile->sFilename] = pFile;
}

void main()
{
    size_t n= gInputFileList.size(); //Using array syntax (iterator syntax also gives identical performance)
    for(size_t i=0; i<n; i++){
        processFile(gInputFileList[i]);
    }
}

The code works correctly, but...

My problem is that when the input size is 1000, it takes 30 minutes - HALF AN HOUR - on Windows/Visual Studio 2008 Express. For the same input, it takes only 40 seconds to run on Linux/gcc!

What could be the problem? The action foo() takes only a very short time to execute, when used separately. Should I be using something like vector::reserve for the map?

EDIT, EXTRA INFORMATION

What foo() does is: 1. it opens the file 2. reads it into memory 3. closes the file 4. the contents of the file in memory is parsed 5. it builds a list of tokens; I'm using a vector for that.

Whenever I break the program (while running the program with the 1000+ files input set): the call-stack shows that the program is in the middle of a std::vector add.

like image 412
sonofdelphi Avatar asked May 14 '10 05:05

sonofdelphi


1 Answers

In the Microsoft Visual Studio, there's a global lock when accessing the Standard C++ Library to protect from multi threading issue in Debug builds. This can cause big performance hits. For instance, our full test code runs on Linux/gcc in 50 minutes, whereas it needs 5 hours on Windows VC++2008. Note that this performance hit does not exist when compiling in Release mode, using the non-debug Visual C++ runtime.

like image 57
Didier Trosset Avatar answered Sep 30 '22 02:09

Didier Trosset