Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to whiten matrix in PCA

I'm working with Python and I've implemented the PCA using this tutorial.

Everything works great, I got the Covariance I did a successful transform, brought it make to the original dimensions not problem.

But how do I perform whitening? I tried dividing the eigenvectors by the eigenvalues:

S, V = numpy.linalg.eig(cov)
V = V / S[:, numpy.newaxis]

and used V to transform the data but this led to weird data values. Could someone please shred some light on this?

like image 746
mabounassif Avatar asked Jul 04 '11 18:07

mabounassif


People also ask

How do you whiten a matrix?

Whitening a data matrix follows the same transformation as for random variables. An empirical whitening transform is obtained by estimating the covariance (e.g. by maximum likelihood) and subsequently constructing a corresponding estimated whitening matrix (e.g. by Cholesky decomposition).

Does PCA Whiten data?

We have used PCA to reduce the dimension of the data. There is a closely related preprocessing step called whitening (or, in some other literatures, sphering) which is needed for some algorithms. If we are training on images, the raw input is redundant, since adjacent pixel values are highly correlated.

What is PCA Whiten?

PCA Whitening is a processing step for image based data that makes input less redundant. Adjacent pixel or feature values can be highly correlated, and whitening through the use of PCA reduces this degree of correlation.

What is whitening normalization?

Normalization of multi-dimensional variables, which we call statistical whitening, not only scales each variance term to 1 , it removes all of the off-diagonal covariance terms. Whitening linearly decorrelates the input dimensions.


1 Answers

If you use python's scikit-learn library for this, you can just set the inbuilt parameter

from sklearn.decomposition import PCA
pca = PCA(whiten=True)
whitened = pca.fit_transform(X)

check the documentation.

like image 53
Shubham Bansal Avatar answered Oct 04 '22 20:10

Shubham Bansal