How do you generate positive definite matrix in pytorch?

Question

I am trying to define Multivariate Gaussian distribution with randomly generated covariance matrix:

psi = torch.zeros(512).normal_(0., 1.).requires_grad_(True)

# Generate random matrix
Sigma_k = torch.rand(512, 512)
# Make it symmetric positive
Sigma_k = Sigma_k * Sigma_k.t()
# Make it definite
Sigma_k.add_(0.001, torch.eye(512)).requires_grad_(True)

multivariate_normal.MultivariateNormal(psi, Sigma_k)

But I end up with getting an exception:

RuntimeError: Lapack Error in potrf : the leading minor of order 2 is not positive definite at /Users/soumith/mc3build/conda-bld/pytorch_1549597882250/work/aten/src/TH/generic/THTensorLapack.cpp:658

What is the proper way to generate positive definite square matrix?

Answer 1

The answer is one should make a dot product of matrix A and it's transpose matrix (A.t()) in order to obtain a positive semi-definite matrix. The last thing is to ensure that it is definite (strictly greater than zero).

With Pytorch:

Sigma_k = torch.rand(512, 512)
Sigma_k = torch.mm(Sigma_k, Sigma_k.t())
Sigma_k.add_(torch.eye(512))

Formal algorithm is described here.

Answer 2

In "make it definite"

tensor.add() does not change tensor, but only returns a changed version. You want to use tensor.add_()