numpy.savetxt saving every element on new line?

Question

with file("features.txt", "w") as outfile:
    # temp_array is an array of shape (500L,)
    np.savetxt(outfile, X=np.array(temp_array, dtype='uint8'), delimiter=",")

I use the above syntax to store around 10,000 arrays in a text file but I notice that all the elements are being stored on a different line.

If I load the text file using numpy.loadtxt, I will get an array instead of a matrix.

I know can use numpy.reshape on the loaded array to convert it into a matrix but in my application the number of rows won't be known to the user who loads it.

How to solve this issue?

Answer 1

If you give np.savetxt a 1-d array, it will treat said array as a column and write each entry on a new line. I can think of two ways around this. The first is to add a new axis to your array to make it a row vector:

x = np.arange(5)
np.savetxt('test.txt', x[np.newaxis], fmt='%d', delimiter=',')

The second approach is to tell np.savetxt to use a different character other than \n for a newline. For example, a space:

np.savetxt('test.txt', x, fmt='%d', newline=' ', delimiter=',')

Both of these result in a file that looks like

0 1 2 3 4

Answer 2

Look at the code for savetxt. It isn't complex. And since you are opening the file yourself, the key part of savetxt is:

for row in X:
    fh.write(asbytes(format % tuple(row) + newline))

In other words, it is taking your array, tweaking its shape a bit - if needed - and then writing it 'row' by 'row', line by line.

fmt is your input parameter replicated by the number of columns:

fmt = [fmt, ] * ncol

Try replicating that writing yourself, if you aren't happy with what savetxt does for you.

If temp_array is already an array, what is:

X=np.array(temp_array, dtype='uint8')

supposed to be doing for you? Why the X= part.

Also you do you use that clip to write many arrays to a file? I can understand repeatedly opening a file in append mode, or performing multiple savetxt with one open file.