Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memmap in MATLAB for huge arrays

I want to create a memmap in MATLAB. In python I could do this by:

ut = np.memmap('my_array.mmap', dtype=np.float64, mode='w+', shape=(140000,3504))

Then I use it as a normal array, the OS ensured my memory never overflowed. How to do this in MATLAB?

From the docs it seems it wants me to create some array in MATLAB first then write it to a file and read using memmap!

Matlab docs are not clear enough: Please provide an example of creating an random array of size (140000,15000) and multiply it some other similar matrix.

like image 671
Abhishek Bhatia Avatar asked Feb 09 '23 18:02

Abhishek Bhatia


1 Answers

You have to create an empty file first, then use memmapfile:

size=[140000,3504];
filesize=0;
datatype='float64';
filename='my_array.dat';
fid=fopen(filename,'w+');
max_chunk_size=1000000;
%fills an empty file
while filesize<prod(size)
    to_write=min(prod(size)-filesize,max_chunk_size);
    filesize=filesize+fwrite(f, zeros(to_write,1), datatype);
end   
fclose(fid);
m = memmapfile(filename,'Format','double', 'Writable',true);
like image 189
Daniel Avatar answered Feb 15 '23 11:02

Daniel