Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Out of memory only by a matrix transpose

I have a cell, Data, it contains three double arrays,

 Data = 

[74003x253 double]    [8061x253 double]    [7241x253 double]

I'm using a loop to read these arrays and perform some functions,

for ii = 1 : 3
    D = Data {ii} ;
    m = mean (D') ;
    // rest of the code
end

Which gets a warning for mean and says:

consider using different DIMENSION input argument for MEAN

However when I change it to,

for ii = 1 : 3
    D = Data {ii}' ;
    m = mean (D) ;
    // rest of the code
end

I get Out of memory error.

Comparing two codes, can someone explain what happens?

It seems that I get the error only with a Complex conjugate transpose (my data is real valued).

like image 754
Rashid Avatar asked Oct 19 '14 17:10

Rashid


1 Answers

To take the mean for the n:th dimension it is possible use mean(D,n) as already stated. Regarding the memory consumption, I did some tests monitoring with the windows resource manager. The output was kind of expected.

When doing the operation D=Data{ii} only minimum memory is consumed since here matlab does no more than copying a pointer. However, when doing a transpose, matlab needs to allocate more memory to store the matrix D, which means that the memory consumption increases.

However, this solely does not cause a memory overflow, since the transpose is done in both cases.

Case 1

Separately inD = Data{ii}';

Case 2

in D = Data {ii}; m = mean(D');

The difference is that in case 2 matlab only creates a temporary copy of Data{ii}' which is not stored in the workspace. The memory allocated is the same in both cases, but in case 1 Data{ii}' is stored in D. When the memory later increases this can cause a memory overflow.

The memory consumption of D is not that bad (< 200 Mb), but the guess is that the memory got high already and that this was enough to give memory overflow.

like image 177
patrik Avatar answered Oct 01 '22 06:10

patrik