Calculate the standard deviation of a gaussian

Question

I have a list of numbers, which when plotted against its length, gives me a gaussian. I would like to calculate the standard deviation on this gaussian, but the value I get (using the np.std() function) is clearly too small (I get something like 0.00143… when it should be something like 8.234...). I think I’ve been calculating the standard deviation on the y-axis and not on the x-axis (which is on what the standard deviation is supposed to be on), but I’m a bit stuck as to how to do that?

I’ve included my code and a pic of the gaussian I'm trying to calculate the std dev on.

#max_k_value_counter counts the number of times the maximum value of k comes up.

max_k_value_counter_sum = sum(max_k_value_counter)
prob_max_k_value = [0] * len(max_k_value_counter)

# Calculate the probability of getting a particular value for k
for i in range(len(max_k_value_counter)):
        prob_max_k_value[i] = float(max_k_value_counter[i]) / max_k_value_counter_sum

print "Std dev on prob_max_k_value", np.std(prob_max_k_value)

# Plot p(k) vs k_max to calculate the errors on k
plt.plot(range(len(prob_max_k_value)), prob_max_k_value)
plt.xlim(0, 200)
plt.xlabel(r"$k$", fontsize=16)
plt.ylabel(r"$p(k)$", fontsize=16)
plt.show()

Answer 1

you are measuring the standard deviation of probabilities not the actual values; Here, is an example, where I draw from true standard normal distribution:

>>> from scipy.stats import norm
>>> xs   = np.linspace(-3, 3, 100)
>>> pdf  = norm.pdf(xs)
>>> prob = pdf / pdf.sum() # these are probabilities
>>> np.std(prob)           # note the very small value below
0.008473522157507624

The correct way here is to use this formula:

to measure variance and then take the square root to get standard deviation; The first term is basically the second moment and the second term is mean squared:

>>> mu   = xs.dot(prob)               # mean value
>>> mom2 = np.power(xs, 2).dot(prob)  # 2nd moment
>>> var  = mom2 - mu**2               # variance
>>> np.sqrt(var)                      # standard deviation
0.98764819824739092

Note that the value we get is very close to 1 which is consistent with the fact that I draw from standard normal;