Calculating inverse of a very large matrix

Question

I am trying to calculate inverse of a very large matrix (11300x21500) in C++. So far I have tried Eigen and Armadillo libraries but both failed at initialization stage, saying that there is not enough memory. Can there be any way to overcome this situation?

Thanks in advance

P.S
I should correct the size of the matrix to 21500x21500. As UmNyobe suggested, this is not a square matrix. It is actually the observation matrix, X, and I am trying to calculate (X^TX)^-1

I have a 8GB memory(in a 64bit system), but I don't think I am making use of all of this memory space. The task manager shows that the memory usage at the time of error is 1GB. Maybe there is a OS command in Windows7 that closes an application when its memory usage exceeds 1GB.

By the way, my original purpose is to run a regression over this observation matrix.

One more thing: most columns in each row of the observation matrix X are zero. Can there be a way to take advantage of this, to limit the memory usage in the inverting operation?

Answer 1

Supposing the matrix is square, what you're probably looking for is an in-place matrix inversion algorithm.

You should check out this.

Answer 2

You cannot inverse a non-square matrix.

http://en.wikipedia.org/wiki/Invertible_matrix

Answer 3

Assuming a (11300 x 11300) Matrix of integer (32 bits), you have

4*(11300^2)/(1024^3) = 0.4757 GB

If you are using double precision then double this number.

If the library is using the Strassen algorithm, which requires additional memory of the same magnitude, then you double the previous number.

So inverting a double-based matrix of this size with Strassen or gaussian will cost you 1.9 GB.