c++ reading file is too slow

Question

I'm trying to to read ~36KB and it would take ~20 seconds to finish this loop:

ifstream input_file;

input_file.open("text.txt");
if( !(input_file.is_open()) )
{
    cout<<"File not found";
    exit(1);
}

std::string line;
stringstream line_stream;   //to use << operator to get words from lines

int lineNum=1;

while( getline(input_file,line) )   //Read file line by line until file ends
{
    line_stream.clear();    //clear stream
    line_stream << line;    //read line
    while(line_stream >> word)  //Read the line word by word until the line ends
    {
        //insert word into a linked list...
    }
    lineNum++;
}
input_file.close();

Any help would be appreciated.

Answer 1

stringstream::clear() does not clear all context inside it. It only resets the error and EOF flags, see http://en.cppreference.com/w/cpp/io/basic_ios/clear.

The result is your line_stream accumulates all previous lines and the inner loop will run words over all the accumulated lines again and again.

So the total time you spend is about O(n^2) compared to O(n) of what you expect it to be.

Instead of using the same object across each line, you could define the new line_stream instance inside the while loop to have a brand new and also empty one. Like this:

fstream input_file;

input_file.open("text.txt");
if( !(input_file.is_open()) )
{
    cout<<"File not found";
    exit(1);
}

std::string line;

int lineNum=1;

while( getline(input_file,line) )   //Read file line by line until file ends
{
    stringstream line_stream;   // new instance, empty line.
    line_stream << line;    //read line
    while(line_stream >> word)  //Read the line word by word until the line ends
    {
        //insert word into a linked list...
    }
    lineNum++;
}
input_file.close();