Why is MATLAB failing to successfully read in binary files?

Question

Matlab is failing to read in the specified number of elements from a file. I have a simple program that needs to read in two files, perform a linear operation on the data and write a combined result to a third file.

My questions are: 1) Why does Matlab fail to read the specified number of elements and 2) is there a workaround for this? Any of your thoughts will be helpful.

Some details on the input files:

• they are large (~18GB)
• they are both the same size (exactly)

Details on the procedure (2-4 are conditioned on an feof check of both files:

1. Open the input and output files for reading and writing (resp.)
2. Read in N floats (N*4 bytes) from each of the input files
3. Perform an operation on the data (say 0.5*(datin1+datin2))
4. Write the result to the output file.

Granted, this is all very simple and in most cases in the past this has worked well. Unfortunately, at some point in the cycle, MATLAB doesn't get all N floats from one of the files and gives a matrix dimension error on step 3.

CODE SNIP:

N = 2048;
fidin1 = fopen('file1.dat','r','l');
fidin2 = fopen('file2.dat','r','l');
fidout = fopen('outfile.dat','w','l');

%# I could do some assertions on the file sizes,
%# but I know they are the same size (w/o question).

while(~feof(fidin1) && ~feof(fidin2))
    datin1 = fread(fidin1,N,'float=>single',0,'l');
    datin2 = fread(fidin2,N,'float=>single',0,'l');

    %# the following line produces an error after 100 
    %# or more iterations in to the procedure
    datout = 0.5*(datin1+datin2);
    fwrite(fidout,datout,'float',0,'l');
end

UPDATE 1 The error message I'm receiving is:

???Error using ==> plus
Matrix dimension must agree.

UPDATE 2 I followed a suggestion and included ferrorchecks after each read and magically the problem went away. So now a modification to my questions: What could be the root of the problem here? Is this simply a timing issue or bug?

Here is a snip of the updated code (showing only a portion of the code). I'm sure there are better ways to do this. Regardless, the addition of these checks allowed Matlab to complete all the reads from each of the files successfully.

    [datin1 count1]= fread(fidin1,N,'float=>single',0,'l');
    [msg errn1]=ferror(fidin1);
    if errn1
        pos1 = ftell(fidin1);
        error('Error at Position %d in file.  %d bytes were read.',...
            pos1,count1);
    end

    [datin2 count2]= fread(fidin2,N,'float=>single',0,'l');
    [msg errn2]=ferror(fidin2);
    if errn2
        pos2 = ftell(fidin2);
        error('Error at Position %d in file.  %d bytes were read.',...
            pos2,count2);
    end

    %# the following line produces an error after 100 
    %# or more iterations in to the procedure
    datout = 0.5*(datin1+datin2);
    fwrite(fidout,datout,'float',0,'l');

Answer 1

Have you specifically looked at the datin1 and datin2 variables at the time the error occurs? Try going to 'Debug-->Stop if Errors/Warnings...' then select 'Always stop if error (dstop if error)'. Run your program and then once it crashes, look at datin1 and datin2. Hopefully that will explain why adding them together is now working.