R - ensemble with neural network?

Question

This is a small sample of my data.frame

    naiveBayesPrediction knnPred5 knnPred10 dectreePrediction logressionPrediction correctClass
1                non-bob        2         2           non-bob    0.687969711847463            1
2                non-bob        2         2           non-bob     0.85851872253358            1
3                non-bob        1         1           non-bob    0.500470892627383            1
4                non-bob        1         1           non-bob     0.77762739066215            1
5                non-bob        1         2           non-bob    0.556431439357365            1
6                non-bob        1         2           non-bob    0.604868385598237            1
7                non-bob        2         2           non-bob    0.554624186182919            1

I have factored everything

   'data.frame':    505 obs. of  6 variables:
     $ naiveBayesPrediction: Factor w/ 2 levels "bob","non-bob": 2 2 2 2 2 2 2 2 2 2 ...
     $ knnPred5            : Factor w/ 2 levels "1","2": 2 2 1 1 1 1 2 1 2 1 ...
     $ knnPred10           : Factor w/ 2 levels "1","2": 2 2 1 1 2 2 2 1 2 2 ...
     $ dectreePrediction   : Factor w/ 1 level "non-bob": 1 1 1 1 1 1 1 1 1 1 ...
     $ logressionPrediction: Factor w/ 505 levels "0.205412826873861",..: 251 415 48 354 92 145 90 123 28 491 ...
     $ correctClass        : Factor w/ 2 levels "1","2": 1 1 1 1 1 1 1 1 1 1 ...

I then tried to ensemble it using neuralnet

ensembleModel <- neuralnet(correctClass ~ naiveBayesPrediction + knnPred5 + knnPred10 + dectreePrediction + logressionPrediction, data=allClassifiers[ensembleTrainSample,])

Error in neurons[[i]] %*% weights[[i]] : requires numeric/complex matrix/vector arguments

I then tried to put in a matrix

m <- model.matrix( correctClass ~ naiveBayesPrediction + knnPred5 + knnPred10 + dectreePrediction + logressionPrediction, data = allClassifiers )

Error in contrasts<-(*tmp*, value = contr.funs[1 + isOF[nn]]) :
contrasts can be applied only to factors with 2 or more levels

I think it must be something to do with the one feature "decistreePrediction" only having 1 level but it only finds one level out of 2 possible outcomes (bob or non-bob) so I have no idea where to go from there.

Answer 1

The neuralnet function requires the 'variables' to be numeric or complex values because it is doing matrix multiplication which requires numeric or complex arguments. This is very clear in the error returned:

Error in neurons[[i]] %*% weights[[i]] : 
  requires numeric/complex matrix/vector arguments

This is also reflected with the following trivial example.

mat <- matrix(sample(c(1,0), 9, replace=TRUE), 3)
fmat <- mat
mode(fmat) <- "character"

# no error
mat %*% mat

# error
fmat %*% fmat
Error in fmat %*% fmat : requires numeric/complex matrix/vector arguments

As a quick demonstration with the actual function I will use the infert dataset which is used as a demo within the package.

library(neuralnet)
data(infert)

# error
net.infert <- neuralnet(case~as.factor(parity)+induced+spontaneous, infert)
Error in neurons[[i]] %*% weights[[i]] : 
  requires numeric/complex matrix/vector arguments

# no error
net.infert <- neuralnet(case~parity+induced+spontaneous, infert)

You can leave correctClass as a factor because it will be converted to a dummy numeric variable anyway but it may be best to also convert it to the respective binary representation.

My suggestions to you are:

1. Convert your factors to the respective binary representations (i.e. 0's and 1's)
2. Leave logressionPrediction as numeric
3. Omit variables that only have 1 value. Including such variables is completely superfluous as no learning can be accomplished with them.