Why is ostringstream faster than ofstream

Question

To write many piece of data to file, I have 2 approaches:

1. Write to ofstream one by one directly

   ofstream file("c:\\test.txt");
   for (int i = 0; i < 10000; ++i)
   {
       file << data[i];
   }
   

2. Write to istringstream first, and then write to ofstream at once

   ostringstream strstream;
   for (int i = 0; i < 10000; ++i)
   {
       strstream << data[i];
   }
   ofstream file("c:\\test.txt");
   file << strstream.str();
   

Not surprisingly, the second approach is faster, in fact, it is 4 times faster than the first approach on my HP7800 machine.

But why? I know ofstream is using filebuf inside, and ostringstream is using stringbuf - as a buffer they should all reside in memory thus should have no difference.

What is the difference under the hood?

Answer 1

Are you using std::endl a lot instead of '\n'? std::endl does two things: it inserts a '\n' into the stream and then flushes the buffer to disk. I've seen code talking a severe performance hit by doing so. (The code ran 5-10 times faster after that was fixed.)
Flushing to a string buffer will be much faster than flushing to the disk, so that would explain your findings.

If that's not the case you might consider is increasing the buffer size:

const std::size_t buf_size = 32768;
char my_buffer[buf_size];
ofstream file("c:\\test.txt");
file.rdbuf()->pubsetbuf(my_buffer, buf_size);

for (int i = 0; i < 10000; ++i)
{
    file << data[i];
}

Answer 2

Disk is slow. Many small writes are more expensive than one large.