What is the fastest way to write a matrix to a text file in Octave?

Question

I have a large matrix (2e6 x 3) which I have to write to a text file.

dlmwrite takes about 230s to achieve this task.

From your experience what is the fastest way to write a large matrix to a text file?

Answer 1

The following applies to MATLAB, but I suggest you try it in Octave. First of all, if you can - transpose the matrix. Here are examples using fprintf and csvwrite (essentially dlmwrite)

A = rand(3, 1e6);
tic;
fid = fopen('data.txt', 'w+');
for i=1:size(A, 1)
    fprintf(fid, '%f ', A(i,:));
    fprintf(fid, '\n');
end
fclose(fid);
toc

tic;
csvwrite('data.txt', A);
toc;

Elapsed time is 1.311512 seconds.
Elapsed time is 2.487737 seconds.

If not transposed, it will take ages indeed. By default, fprintf flushes the buffer after every call. You can try to use W instead of w to open the file, but it does not improve the situation here too much.

Answer 2

Did you try this? I'm not sure about it's speed as compared to dlmwrite.

a = [1 2;3 4];
save temp.txt a;

Answer 3

Having a variable data you can save it in a text format with space separated values (including a header):

save out.txt data

The header can be simply removed using basic Unix command tail e.g. (on any Linux/Mac OS):

tail -n +6 out.txt > file.csv