finding eigenvector using linalg

Question

I am using the linalg.eig from numpy to find the eigenvalues and vectors of matrices. the matrix below has a single eigenvector of the form (t,0). but python is giving me different results:

>>> a = np.matrix('2. 0. ; 1. 2.')
>>> print np.linalg.eig(a)
(array([ 2.,  2.]), matrix([[  0.00000000e+00,   4.44089210e-16],
        [  1.00000000e+00,  -1.00000000e+00]]))

What is wrong?

Answer 1

What would we expect?

First off, (t, 0) is not an eigenvector of your input matrix:

   | 2.  0.| x |t|  ==> [2t, t]
   | 1.  2.|   |0|

Contrast that with:

   | 2.  0.| x |0|  ==> [0, 2t] == 2 * [0, t]
   | 1.  2.|   |t|

Okay, val = 2, vec = [0, 1] would make sense, as would any multiplier thereof.

---

Rounding Error

Next, keep in mind that eig uses an iterative approximate solution, as analytic solutions for eigenvectors aren't possible for >3x3 inputs. Therefore, you can ignore the fact that one value in your eigenvectors isn't exactly zero.

Your output is basically:

eigs = [2, 2]
vecs = [[0,  0],
        [1, -1]]

Note that the individual eigenvectors of eig are in the columns, not the rows. In other words, we get two identical eigenvalues of 2 and two eigenvectors of [0, 1] and [0, -1].

---

There Can Be Only One!

As @WarrenWeckesser pointed out, this is a defective matrix which only has one eigenvector despite being 2x2.

Therefore, mathematically we'd expect a val = 2, vec = [0, 1]. Instead, we also get another pair: val = 2, vec = [0, -1].

So we expected one eigenvector and got two... Is that even possible?

Wait, We Got Two of the Same Thing!?

As a sanity check, note that [0, -1] will also be an eigenvector of your 2x2 input:

   | 2.  0.| x | 0|  ==> [0, -2t] == 2 * [0, -1]
   | 1.  2.|   |-t|

Of course, this is equivalent to val = -2, vec = [0, 1], so it's the same eigenvector.

So why does numpy.linalg.eig give two output eigenvectors [0, -1] and [0, 1] with identical eigenvalues of 2? They're the same thing!

In a nutshell, np.linalg.eig of an MxM matrix is guaranteed to always return M eigenvalues and vectors.

However, we can do a bit better than that. If that was the only rule, it could just find one and then return various multiples of it.

---

How Does eig Choose Which Eigenvectors/values to Return?

In general, there are an infinite number of eigenvectors for any given matrix, but they'll all have the relationship A * x = lambda * x.

Therefore, to be have a program give a useful output, we need to place some restrictions. Otherwise, it would need to return an infinite number of different but very similar results.

The key is how the output of np.linalg.eig is defined:

1. The number of output eigenvectors is equal to the size of the input (i.e. 3 or a 3x3, 4 for a 4x4, etc).
2. The eigenvectors will have unit length.
3. The output eigenvectors will be orthogonal, if possible
4. If there aren't the appropriate number of orthogonal eigenvectors, the eigenvalues for the linearly dependent eigenvectors will be identical.

---

Following the Rules

Okay, so we're going to get 2 eigenvalues and two eigenvectors as the input is 2x2, and we'll get orthogonal eigenvectors if possible. Furthermore, the eigenvectors will have unit length, limiting which eigenvalues we get.

However, in this case there aren't two independent eigenvectors.

What we're seeing is the fourth rule above coming into play. The documentation states:

The array v of eigenvectors may not be of maximum rank, that is, some of the columns may be linearly dependent, although round-off error may obscure that fact. If the eigenvalues are all different, then theoretically the eigenvectors are linearly independent.

We know that 2 eigenvectors are guaranteed to be produced. From the text above we can infer that if they're actually the same vector, this will be indicated by making the eigenvalues identical.

---

So in this case, where we have a 2x2 matrix with one eigenvalue/vector of 2 and [0, 1]:

If the eigenvalues must be identical and the eigenvector must have a unit length of one, then the only other valid combination is val = 2, vec=[0, -1].