What is the difference between cholesky in numpy and scipy?

Question

I use Cholesky decomposition to sample random variables from multi-dimension Gaussian, and calculate the power spectrum of the random variables. The result I get from numpy.linalg.cholesky always has higher power in high frequencies than from scipy.linalg.cholesky.

What are the differences between these two functions that could possibly cause this result? Which one is more numerically stable?

Here is the code I use:

n = 2000

m = 10000

c0 = np.exp(-.05*np.arange(n))

C = linalg.toeplitz(c0)

Xn = np.dot(np.random.randn(m,n),np.linalg.cholesky(C))

Xs = np.dot(np.random.randn(m,n),linalg.cholesky(C))

Xnf = np.fft.fft(Xn)

Xsf = np.fft.fft(Xs)

Xnp = np.mean(Xnf*Xnf.conj(),axis=0)

Xsp = np.mean(Xsf*Xsf.conj(),axis=0)

Answer 1

scipy.linalg.cholesky is giving you the upper-triangular decomposition by default, whereas np.linalg.cholesky is giving you the lower-triangular version. From the docs for scipy.linalg.cholesky:

cholesky(a, lower=False, overwrite_a=False)
    Compute the Cholesky decomposition of a matrix.

    Returns the Cholesky decomposition, :math:`A = L L^*` or
    :math:`A = U^* U` of a Hermitian positive-definite matrix A.

    Parameters
    ----------
    a : ndarray, shape (M, M)
        Matrix to be decomposed
    lower : bool
        Whether to compute the upper or lower triangular Cholesky
        factorization.  Default is upper-triangular.
    overwrite_a : bool
        Whether to overwrite data in `a` (may improve performance).

For example:

>>> scipy.linalg.cholesky([[1,2], [1,9]])
array([[ 1.        ,  2.        ],
       [ 0.        ,  2.23606798]])
>>> scipy.linalg.cholesky([[1,2], [1,9]], lower=True)
array([[ 1.        ,  0.        ],
       [ 1.        ,  2.82842712]])
>>> np.linalg.cholesky([[1,2], [1,9]])
array([[ 1.        ,  0.        ],
       [ 1.        ,  2.82842712]])

If I modify your code to use the same random matrix both times and to use linalg.cholesky(C,lower=True) instead, then I get answers like:

>>> Xnp
array([ 79621.02629287+0.j,  78060.96077912+0.j,  77110.92428806+0.j, ...,
        75526.55192199+0.j,  77110.92428806+0.j,  78060.96077912+0.j])
>>> Xsp
array([ 79621.02629287+0.j,  78060.96077912+0.j,  77110.92428806+0.j, ...,
        75526.55192199+0.j,  77110.92428806+0.j,  78060.96077912+0.j])
>>> np.allclose(Xnp, Xsp)
True