caret train method not working (something is wrong for all accuracy results) for outcomes with >2 categories

Question

Hi I know someone asked similar issues before but no clear answer yet (or I tried their solution without success: Caret error using GBM, but not without caret Caret train method complains Something is wrong; all the RMSE metric values are missing )

I tried to use caret training methods to predict the categorical outcomes (online data examples below)

library(mlbench)
data(Sonar)
str(Sonar[, 1:10])

library(caret)
set.seed(998)

Sonar$rand<-rnorm(nrow(Sonar))  ##to randomly create the new 3-category outcome
table(Sonar$rand)
Sonar$Class_new<-ifelse(Sonar$Class=="R","R",ifelse(Sonar$rand>0,"M","H"))
table(Sonar$Class_new)

fitControl <- trainControl(## 10-fold CV
                           method = "repeatedcv",
                           number = 10,
                           ## repeated ten times
                           repeats = 10)

inTraining <- createDataPartition(Sonar$Class_new, p = .75, list = FALSE)
training <- Sonar[ inTraining,]
testing  <- Sonar[-inTraining,]

gbmFit1 <- train(Class_new ~ ., data = training,
                 method = "gbm",
                 trControl = fitControl,
                 verbose = FALSE)

Whenever I used the new class variable (Class_new) which has 3 categories, rather than 2 categories in original Class variable, I got the warnings below. It runs fine with 2 category outcome variables. And it is the same case regardless of the train methods (I tried rf, gbm, svm, all the same)

Something is wrong; all the Accuracy metric values are missing:

    Accuracy       Kappa    
 Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA  
 NA's   :9     NA's   :9    

Error in train.default(x, y, weights = w, ...) : Stopping
In addition: Warning messages:
1: In train.default(x, y, weights = w, ...) :
The metric "RMSE" was not in the result set. Accuracy will be used instead.
2: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.

Any help on this is greatly appreciated!

Answer 1

You need to convert the newly created Class_new to a factor, as follows:

Sonar$Class_new<-ifelse(Sonar$Class=="R","R",ifelse(Sonar$rand>0,"M","H"))
Sonar$Class_new <- factor(Sonar$Class_new)

Also, you may want to remove the variables Class and rand from your training and testing data sets. You can do somthing like:

training <- Sonar[ inTraining, !(names(Sonar) %in% c("Class", "rand"))]
testing <- Sonar[-inTraining, !(names(Sonar) %in% c("Class", "rand"))]

Answer 2

I had allowParallel = TRUE in the train function and the machine I was working on did not have multiple cores. After I commented that statement, I did not get the error.