Reading data into MATLAB from a textfile

Question

I have a textfile with the following structure:

1999-01-04
1,100.00
1,060.00
1,092.50
0
6,225
1,336,605
37
1999-01-05 
1,122.50
1,087.50
1,122.50
0
3,250
712,175
14
...

The file contains repeated sets of eight values (a date followed by seven numbers, each on their own line).

I want to read it into MATLAB and get the values into different vectors. I've tried to accomplish this with several different methods, but none have worked - all output some sort of error.

In case it's important, I'm doing this on a Mac.

Answer 1

EDIT: This is a shorter version of the code I previously had in my answer...

If you'd like to read your data file directly, without having to preprocess it first as dstibbe suggested, the following should work:

fid = fopen('datafile.txt','rt');
data = textscan(fid,'%s %s %s %s %s %s %s %s','Delimiter','\n');
fclose(fid);
data = [datenum(data{1}) cellfun(@str2double,[data{2:end}])]';

The above code places each set of 8 values into an 8-by-N matrix, with N being the number of 8 line sets in the data file. The date is converted to a serial date number so that it can be included with the other double-precision values in the matrix. The following functions (used in the above code) may be of interest: TEXTSCAN, DATENUM, CELLFUN, STR2DOUBLE.

Answer 2

I propose yet another solution. This one is the shortest in MATLAB code. First using sed, we format the file as a CSV file (comma seperated, with each record on one line):

cat a.dat | sed -e 's/,//g ; s/[ \t]*$/,/g' -e '0~8 s/^\(.*\),$/\1\n/' | 
            sed -e :a -e '/,$/N; s/,\n/,/; ta' -e '/^$/d' > file.csv

Explanation: First we get rid of the thousands comma seperator, and trim spaces at the end of each line adding a comma. But then we remove that ending comma for each 8th line. Finally we join the lines and remove empty ones.

The output will look like this:

1999-01-04,1100.00,1060.00,1092.50,0,6225,1336605,37
1999-01-05,1122.50,1087.50,1122.50,0,3250,712175,14

Next in MATLAB, we simply use textscan to read each line, with the first field as a string (to be converted to num), and the rest as numbers:

fid = fopen('file.csv', 'rt');
a = textscan(fid, '%s %f %f %f %f %f %f %f', 'Delimiter',',', 'CollectOutput',1);
fclose(fid);

M = [datenum(a{1}) a{2}]

and the resulting matrix M is:

  730124     1100     1060   1092.5    0   6225   1336605    37
  730125   1122.5   1087.5   1122.5    0   3250    712175    14