Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Having trouble converting r chisquare goodness of fit test code to python equivalent

UCLA has this great site for statistical tests

https://stats.idre.ucla.edu/r/whatstat/what-statistical-analysis-should-i-usestatistical-analyses-using-r/#1sampt

but the code is all in R. I am trying to convert the code to Python equivalents but it is not a straightforward process for some like the chi-square goodness of fit. Here is the R version:

hsb2 <- within(read.csv("https://stats.idre.ucla.edu/stat/data/hsb2.csv"), {
  race <- as.factor(race)
  schtyp <- as.factor(schtyp)
  prog <- as.factor(prog)
})
chisq.test(table(hsb2$race), p = c(10, 10, 10, 70)/100)

My Python attempt is this:

import numpy as np
import pandas as pd
from scipy import stats

df = pd.read_csv("https://stats.idre.ucla.edu/stat/data/hsb2.csv")
# convert to category
df["race"] = df["race"].astype("category")

t_race = pd.crosstab(df.race, columns = 'race')
p_tests = np.array((10, 10, 10, 70))
p_tests = ptests/100
# tried this
stats.chisquare(t_race, p_tests)
# and this
stats.chisquare(t_race.T, p_tests)

but neither stats.chisquare output comes close to the R version. Can anybody steer me in the right direction? TIA

like image 538
Rufus McGee Avatar asked Nov 27 '25 18:11

Rufus McGee


1 Answers

chisq.test takes a vector of probabilities; stats.chisquare takes the expected frequencies (docs).

> results = chisq.test(c(24, 11, 20, 145), p=c(0.1, 0.1, 0.1, 0.7))
> results

    Chi-squared test for given probabilities

data:  c(24, 11, 20, 145)
X-squared = 5.028571429, df = 3, p-value = 0.169716919

vs.

In [49]: obs = np.array([24, 11, 20, 145])

In [50]: prob = np.array([0.1, 0.1, 0.1, 0.7])

In [51]: stats.chisquare(obs, f_exp=obs.sum() * prob)
Out[51]: Power_divergenceResult(statistic=5.0285714285714285, pvalue=0.16971691923343338)
like image 128
DSM Avatar answered Nov 29 '25 08:11

DSM



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!