How to See the details of Models Trained by Autogluon?

Question

I am just trying my hand in automated model training via Amazon Autogluon based on Mxnet, especially, TabularPrediction. It is indeed training several models such as Catboost, LightGBM etc. based on the training data and dumping the models into a specified directory as pickle files.

However, the whole thing is still too much of a blackbox to me. In particular, I am looking for ways so that

• I can make some adjustments to the models trained by autogluon
• I can inspect the individual models, specifically, check their hyper parameters, or export them as standalone models
• I can adjust the weights of different models while predicting online, including setting the weights to zero (effectively ignoring them) for some models.
• Include my own model e.g. a neural network trained by Pytorch or an SKLearn model in the framework so that together they behave the same way as the autogluon models. May be this can be achieved by doing another additional layer of ensemble and creating my own class. But still, that sounds like a hack.

Are these things possible? Some reference, assuming that Amazon does provide APIs for these would be greatly helpful.

Answer 1

Note: These answers apply to AutoGluon 0.0.9

Regarding adjustments to models, this can be done by specifying hyperparameters in the fit() call. The ability to adjust models post-fit is a feature which is not implemented yet, but is planned for future releases.

Regarding inspection of models: predictor.info() will show all model hyperparameters and various other information.

To export them as stand-alone models, call predictor.delete_models(models_to_keep='my_model', dry_run=False). This will delete all models except the particular model you wish to keep, effectively treating the predictor as a single model. Note that it still must use the predictor API, as the model requires data preprocessing present outside of its pickle file.

Regarding adjusting model weights, this is an open feature request that is intended to be added in a future release: https://github.com/awslabs/autogluon/issues/486

Regarding including custom models, the best way without extending AutoGluon at present is to treat the output of AutoGluon as a single model and utilize your own ensembling code. Alternatively, you can inherit from the AbstractModel class of AutoGluon to wrap your model, but this is intended for developers and is not well documented.