Is there a way to set up a multi-hidden layer neural network with the mlp method in the caret package?

Question

The mlp method in package caret calls the mlp function in RSNNS. In the RSNNS package, I can set up as many hidden layers in the neural net as I like by setting the size parameter, e.g.

data(iris)

#shuffle the vector
iris <- iris[sample(1:nrow(iris),length(1:nrow(iris))),1:ncol(iris)]

irisValues <- iris[,1:4]
irisTargets <- decodeClassLabels(iris[,5])
#irisTargets <- decodeClassLabels(iris[,5], valTrue=0.9, valFalse=0.1)

iris <- splitForTrainingAndTest(irisValues, irisTargets, ratio=0.15)
iris <- normTrainingAndTestSet(iris)

model <- mlp(iris$inputsTrain, iris$targetsTrain, size=c(5,7), learnFuncParams=c(0.1), 
             maxit=50, inputsTest=iris$inputsTest, targetsTest=iris$targetsTest)

Will set up a neural net with two hidden layers of 5 and 7 nodes respectively. I want to use the caret package because it has functionality for doing parameter/model searches, as well as parallel implementations for a cluster. In caret, when I look up the method, it can only be tuned with one parameter, size, e.g.

data(iris)

mlpGrid <- data.frame(.size=3)
model2<-caret::train(Species~. , iris, method='mlp', tuneGrid=mlpGrid)

Sets up a neural net with a 3-node single hidden layer.

I've tried adding other columns to mlpGrid and such, but caret doesn't seem to allow for adding a second (or more) hidden layer.

Answer 1

You should use caret's "mlpML" method insted of "mlp". It do uses mlp function from RSNNS, but you are able to define the number of neurons per hidden layer separately. For instance, the following code should do the work. You define your customized grid with the definition of your layers, each layer (1, 2, and 3) and how many neurons per layer.

mlp_grid = expand.grid(layer1 = 10,
                       layer2 = 10,
                       layer3 = 10)

mlp_fit = caret::train(x = train_x, 
                y = train_y, 
                method = "mlpML", 
                preProc =  c('center', 'scale', 'knnImpute', 'pca'),
                trControl = trainControl(method = "cv", verboseIter = TRUE, returnData = FALSE),
                tuneGrid = mlp_grid)

Given the verboseIter=TRUE it shows that the values were indeed applied

+ Fold01: layer1=10, layer2=10, layer3=10 
+ Fold02: layer1=10, layer2=10, layer3=10 
+ Fold03: layer1=10, layer2=10, layer3=10
...