I am trying to run a simple logistic regression function. I have 4 columns named x1, x2, x3, and x4. x4 has a column that has only zeros and ones. So, I am using this as my dependent variable. To predict the dependent variable, I am using the independent variables x1, x2, and x3. Is my syntax off or how can I properly complete a logistic regression on my data while maintaining the R syntax that Statsmodels.api provides?
The following is my code:
import pandas as pd
import statsmodels.formula.api as smf
df = pd.DataFrame({'x1': [10, 11, 0, 14],
'x2': [12, 0, 1, 24],
'x3': [0, 65, 3, 2],
'x4': [0, 0, 1, 0]})
model = smf.logit(formula='x4 ~ x1 + x2 + x3', data=df).fit()
print(model)
The following is my error:
statsmodels.tools.sm_exceptions.PerfectSeparationError: Perfect separation detected, results not available
I understand what it means but I do not understand how I can avoid this issue. What values are needed to confirm a successful logistic regression algorithm and is my syntax correct and is there a better way to solve what I did (with the R syntax)?
I may be misunderstanding the question, but the syntax seems fine -- though I think you want print(model.summary())
rather than print(model)
. The issue is that your sample size is too small.
For example, this works:
import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
np.random.seed(2)
n=100
df = pd.DataFrame({'x1':np.random.randn(n),
'x2': np.random.randn(n),
'x3': np.random.randn(n),
'x4': np.random.randint(0,2,n)})
model = smf.logit(formula='x4 ~ x1 + x2 + x3', data=df).fit()
print(model.summary())
Changing to n=10
yields the following message under the summary table:
Possibly complete quasi-separation: A fraction 0.40 of observations can be perfectly predicted. This might indicate that there is complete quasi-separation. In this case some parameters will not be identified.
Changing to n=5
yields
PerfectSeparationError: Perfect separation detected, results not available
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With