In some (e.g. machine learning) libraries, we can find log_prob
function. What does it do and how is it different from taking just regular log
?
For example, what is the purpose of this code:
dist = Normal(mean, std)
sample = dist.sample()
logprob = dist.log_prob(sample)
And subsequently, why would we first take a log and then exponentiate the resulting value instead of just evaluating it directly:
prob = torch.exp(dist.log_prob(sample))
Categorical. Creates a categorical distribution parameterized by either probs or logits (but not both). It is equivalent to the distribution that torch.
As your own answer mentions, log_prob
returns the logarithm of the density or probability. Here I will address the remaining points in your question:
log
? Distributions do not have a method log
. If they did, the closest possible interpretation would indeed be something like log_prob
but it would not be a very precise name since if begs the question "log of what"? A distribution has multiple numeric properties (for example its mean, variance, etc) and the probability or density is just one of them, so the name would be ambiguous.The same does not apply to the Tensor.log()
method (which may be what you had in mind) because Tensor
is itself a mathematical quantity we can take the log of.
p
and q
, then you can directly compute log(p * q)
as log(p) + log(q)
, avoiding intermediate exponentiations. This is more numerically stable (avoiding underflow) because probabilities may become very close to zero while their logs do not. Addition is also more efficient than multiplication in general, and its derivative is simpler. There is a good article about those topics at https://en.wikipedia.org/wiki/Log_probability.Part of the answer is that log_prob
returns the log of the probability density/mass function evaluated at the given sample value.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With