using eigenvalues to test for singularity: identifying collinear columns

Question

I am trying to check if my matrix is singular using the eigenvalues approach (i.e. if one of the eigenvalues is zero then the matrix is singular). Here is the code:

z <- matrix(c(-3,2,1,4,-9,6,3,12,5,5,9,4),nrow=4,ncol=3) 
eigen(t(z)%*%z)$values

I know the eigenvalues are sorted in descending order. Can someone please let me know if there is a way to find out what eigenvalue is associated to what column in the matrix? I need to remove the collinear columns.

It might be obvious in the example above but it is just an example intended to save you time from creating a new matrix.

Answer 1

Example:

z <- matrix(c(-3,2,1,4,-9,6,3,12,5,5,9,4),nrow=4,ncol=3)
m <- crossprod(z) ## slightly more efficient than t(z) %*% z

This tells you that the third eigenvector corresponds to the collinear combinations:

ee <- eigen(m)
(evals <- zapsmall(ee$values))
## [1] 322.7585 124.2415   0.0000

Now examine the corresponding eigenvectors, which are listed as columns corresponding to their respective eigenvalues:

   (evecs <- zapsmall(ee$vectors))
   ## [1,] -0.2975496 -0.1070713  0.9486833
   ## [2,] -0.8926487 -0.3212138 -0.3162278
   ## [3,] -0.3385891  0.9409343  0.0000000

The third eigenvalue is zero; the first two elements of the third eigenvector (evecs[,3]) are non-zero, which tells you that columns 1 and 2 are collinear.

Here's a way to automate this test:

   testcols <- function(ee) {
       ## split eigenvector matrix into a list, by columns
       evecs <- split(zapsmall(ee$vectors),col(ee$vectors))
       ## for non-zero eigenvalues, list non-zero evec components
       mapply(function(val,vec) {
           if (val!=0) NULL else which(vec!=0)
       },zapsmall(ee$values),evecs)
   }

testcols(ee)
##  [[1]]
## NULL
## [[2]]
## NULL
## [[3]]
## [1] 1 2

Answer 2

You can use tmp <- svd(z) to do a svd. The eigenvalues are then saved in tmp$d as a diagonal matrix of eigenvalues. This works also with a non square matrix.

> diag(tmp$d)
         [,1]     [,2]         [,3]
[1,] 17.96548  0.00000 0.000000e+00
[2,]  0.00000 11.14637 0.000000e+00
[3,]  0.00000  0.00000 8.787239e-16