meassure the entropy of a point-cloud

Question

I have a point-clould of which I would like to meassure (/approximate) the entropy (density-entropy). The entropy can only be computed if I assign a probability to each point in the point-cloud. What is the common way to do this?

Note:

One Idea I have is to compute the density with a kernel density estimation (wiki):

Is there a common approach, or should I just use kde?

Answer 1

The kernel density estimator is a well established classic https://jakevdp.github.io/PythonDataScienceHandbook/05.13-kernel-density-estimation.html

If you have lots of points or need to be especially fast, a histogram may be a worth a try. The implementation can be quite simple, e.g. python

import numpy as np
hist = np.histogramdd(np.array(data), bins=n_bins)[0]
hist /= hist.sum()
hist = hist.flatten()
hist = hist[hist.nonzero()]
entropy = -0.5 * np.sum(hist * np.log2(hist))

Alternatively, if you are working in high dimensions a gaussian-mixture model may be a helpful proxy. https://scikit-learn.org/stable/modules/density.html