results from lm using caret train

Question

I am using the caret R package as an extremely convenient wrapper for modeling. Although this is an odd use, I am having some trouble extracting results from a model when using model type = "lm", and cross-validation of "none". See below for example:

library(caret)
## Make data
Xs <- matrix(rnorm(300*20), nrow = 300, ncol = 20)
Yvec <- rnorm(300)
## Make traincontrol, cv of "none"
tcontrol <- trainControl(method = "none")
## Fit lm model using train
fit <- train(x= Xs, y = Yvec, method = "lm", metric = "RMSE", trControl = tcontrol)

fit$results
[1] RMSE      Rsquared  parameter
<0 rows> (or 0-length row.names)

Any ideas why the fit$results are empty? For all other models and cv types this seems to work. e.g. using 2-fold CV:

tcontrol2 <- trainControl(method = "cv", number = 2)
fit2      <- train(x= Xs, y = Yvec, method = "lm", metric = "RMSE", trControl = tcontrol2)
fit2$results
  parameter     RMSE     Rsquared      RMSESD   RsquaredSD
1      none 1.023666 0.0008921194 0.006499062 0.0003463973

I appreciate that this is a contrived example, but this model and cv method is just one combination of a much larger number that I am testing (and so the caret wrapper is ideal).

Answer 1

Actually, I think I may have a solution. When there is no resampling, I believe you can still get the fitted values from the output of train. If true, this would mean that the RMSE/Rsq for a single model (i.e. no cross-validation/resampling) is just:

> caret::RMSE(pred = fit$finalModel$fitted.values, obs = Yvec)
[1] 0.9348365
> caret::R2(pred = fit$finalModel$fitted.values, obs = Yvec)
[1] 0.04692012

And this R^2 corresponds to what you would get if you just did lm manually:

> dat <- cbind(Yvec, Xs) %>% as.data.frame()
> lm(Yvec ~., data=dat) %>% summary

Call:
lm(formula = Yvec ~ ., data = dat)

...

...Multiple R-squared:  0.04692...

Think this is case closed, but will leave this up in case others find it helpful, and I would still love to hear if anyone can confirm/refute this.