C++ write to csv, performance

Question

This is probably a simple question, but I have not been able to find specific information on this, or atleast information in a readable format. Most of the information I have found relates to reading data from a .csv.

I am have a function that have to save data to a .csv file. This is not an ideal format in a performance perspective, but let us assume that this cannot change. My data is stored in a r x c x s data structure and has to be outputed in the form r,c,s,value and saved to the .csv. At the moment i have:

char delimiter = ',';
ofstream ofs(file, ofstream::out);

for (int r = 0; r < P.n_rows; r++)
{
    for (int c = 0; c < P.n_cols; c++)
    {
        for (int s = 0; s < P.n_slices; s++)
        {
            ofs << r + 1 << delimiter << c + 1 << delimiter << s + 1 << delimiter << P(c, s, s) << endl;
            count++;
        }
    }
}
ofs.close();

For a data structure of size 100 x 100 x 50 this take roughly 6 sec, which I fell is an necessary long time. I would much appreciate if you could provide some information on how to speed this up.

Answer 1

You should note that endl is more than a newline - it actually flushes data to the disk.

Inserts a newline character into the output sequence os and flushes it as if by calling os.put(os.widen('\n')) followed by os.flush().

This might slow down things considerably. You should try replacing it with a newline.

Answer 2

As was stated (and accepted) above, dropping endl reduces the time by 50-60% (in my case - from 7 seconds to 2 seconds, more than 70%).

However, there is still room for improvement: the general stream formatting. The following code further reduces run time by another 75%, to 500 ms:

int a[100][100][50];
int main(int argc, char** argv)
{
    char buff[64];
    memset(a, 1, 100 * 100 * 50 * sizeof(int));
    int count(0);
    char delimiter = ',';
    auto start = std::chrono::steady_clock::now();
    std::ofstream ofs("test.csv", std::ofstream::out);
    for (int r = 0; r < 100; r++)
    {
        for (int c = 0; c < 100; c++)
        {
            for (int s = 0; s < 50; s++)
            {
                sprintf_s(buff, "%d,%d,%d,%d\n", r, c, s, a[c][r][s]);
                ofs << buff;
                //ofs << r + 1 << delimiter << c + 1 << delimiter << s + 1 << delimiter << a[c][r][s] << '\n';
                count++;
            }
        }
    }
    ofs.close();
    auto end = std::chrono::steady_clock::now();
    std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << " ms" << endl;
    return count;
}