Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Ridge regression in glmnet in R; Calculating VIF for different lambda values using glmnet package

I have a set of multicollinear variables and I'm trying to use ridge regression to tackle that. I am using the glmnet package in R with alpha = 0 (for ridge regression).

library(glmnet)

I have a sequence of lambda values, and I am choosing the best lambda value through cv.glmnet

lambda <- 10^seq(10, -2, length = 100)

-- creating model matrix and assigning the y variable

x <- model.matrix(dv ~ ., datamatrix) [,-1]
y <- datamatrix$dv

-- Using cross validation to determine the best lambda and predicting y using that lambda value

ridge.mod <- glmnet(x, y, alpha = 0, lambda = lambda)
cv.out <- cv.glmnet(x, y, alpha = 0)
ridge.pred <- predict(ridge.mod, s = cv.out$lambda.min, newx = x)

I am able to successfully do till this point, but I have to also check for the VIF for this particular lambda value to ensure that the coefficients have stabilized and the multicollinearity is controlled. But I am not sure how to check for VIF in GLMNET since the usual vif() function throws this error.

Error in vcov.default(mod) : there is no vcov() method for models of class elnet, glmnet

Could you please help me identify if there is anything wrong in my approach or how to solve this issue?

Is VIF not applicable for validation in GLMNET?

Thanks in advance.

like image 215
srt Avatar asked Jul 01 '17 14:07

srt


2 Answers

Hadi Regression Analysis by Examples (p295) has the following ridge regression definition of the VIF. Z is the standardized version of the covariate matrix.

VIF for Ridge

like image 155
kaz_yos Avatar answered Sep 28 '22 20:09

kaz_yos


VIF is a property of set of independent variables only. It doesn't matter what dependent variable is and what kind of model you use (linear regression, generalized model) as long as it doesn't change indeperndent variables (as e.g. additive model does). See vif function from car package. So, VIF applied to elastic net regression, won't tell you if you have dealt with multicollinearity. It can just tell you that there was a multicollinearity to deal with.

like image 35
Łukasz Deryło Avatar answered Sep 28 '22 18:09

Łukasz Deryło