The fastest way to save large data to a file

Question

I do some numerical calculation in Java, C# and C++. Some of them save a lot of data (to the text file). What is the fastest way to do it?

C++.

ofstream file;
file.open(plik);
for(int i=0;i<251;i++){
    for(int j=0;j<81;j++)
        file<<(i-100)*0.01<<" "<<(j-40)*0.01<<" "<<U[i][j]<<endl;
    file<<endl;
}

Which I assume is very fast ( am I right?:) )

Java

void SaveOutput(double[][] U, String fileName) throws IOException
{
    PrintWriter tx = new PrintWriter(new FileWriter(fileName));
    for(int i=0;i<251;i++)
    {
        for(int j=0;j<81;j++)
        {
            tx.println(String.format("%e %e %e ",(i - 100) * dz, (j - 40) * dz, U[i][j]));
        }
        tx.println();
    }
    tx.close();
}

The C# example is similar.

and here is what bothers me. I make a String object for each line (a lot of garbage). In this example it is not that much but sometimes I have 10 000 000 lines. This leads me to questions:

1. Can the c++ example be faster?
2. Should I use StringBuilder for Java or maybe it also bad due to number of lines
3. There is any other way or library?
4. What about C#?

Thank you

Answer 1

Profile it. Run the code, time it, see how long it takes. If the amount of time it takes is acceptable, use it. If not, figure out what piece is taking a long amount of time to run, and optimize it.

• Make it right.
• Make it fast.

That order. (Some people add "make it run/build" before those two...)

That said, I've actually run metrics on this sort of thing before. The short of it: You're waiting for disk, and disk is ungodly slow. It doesn't matter if you're writing in C or C++, or Java, they're all waiting for the hard disk.

Here's a previous post that I did on various I/O methods in C. Not exactly what you're looking for, but might be informative.