Unable to load non-winning models in automl() use case

Question

I'm using automl function with code snippet shown below

h2o.init()
h2o_train = as.h2o(train)
h2o_test = as.h2o(test)
aml <- h2o.automl(x=x, y=y, training_frame=h2o_train, leaderboard_frame=h2o_test)
print(aml@leaderboard)  # view top models
print(getParms(aml@leader))  #  get related info for top1 model 

After read through the doc, I couldn't find how to load the results for other models, the leaderboard shows their model_id. It would be valuable if we can load those models, or at least see their parameters.

Answer 1

You can get a list of all models Id using the following:

> aml@leaderboard 

Note the output will be something as below:

                                               model_id      auc  logloss
1    DeepLearning_grid_0_AutoML_20171205_070022_model_1 0.808806 0.536941
2             GLM_grid_0_AutoML_20171205_070022_model_0 0.808672 0.524783
3 StackedEnsemble_BestOfFamily_0_AutoML_20171205_070022 0.797148 0.541090
4    DeepLearning_grid_0_AutoML_20171205_070022_model_2 0.793247 0.654405
5    StackedEnsemble_AllModels_0_AutoML_20171205_070022 0.788943 0.545078
6                 DeepLearning_0_AutoML_20171205_070022 0.783562 0.570281

After that you can use h2o.getModel() API to get any of the model as below:

> aml6 = h2o.getModel("DeepLearning_0_AutoML_20171205_070022")
> aml6

The above will give you the access to model = 6 from the AML leaderboard. Any of H2O Model API will work once you have access to model using the model_id from getModel() API.

Answer 2

To get any model you can do m <- h2o.getModel(model_id). The model_id can be any model id from the leaderboard.

To see the list of non-default parameters, you can do h2o.getModel(model_id)@parameters or h2o.getModel(model_id)@allparameters to see all parameters, including default values.

Hope this helps.

-Navdeep