I am training a ML.Net machine learning model. I can train it and predict from it, and save/load it from the disk. But I need to be able to load it off the disk, then retrain it, or add to it with new information to improve it over time.
Does anyone know if this is possible? I have not found anything in the MS docs of how to do it, but it's a pretty standard thing for ML, so I'd be surprised if it's not possible.
Thanks
Error Rate Based Drift Detection In this strategy, when we observe a significant dip in model performance, we retrain our model. The threshold for retraining should be determined based on the performance expectations set during model development.
Why is model retraining necessary? As the business environment and data change, the prediction accuracy of your ML models will begin to decrease compared to their performance during testing. This is called model drift and it refers to the degradation of ML model performance over time.
If the market is fast changing, you should even consider retraining periodically based on new data only.
If you do end up looking into ML.NET, I recommend looking at the ML.NET Model Builder - there's a really straightforward tutorial here. Essentially you can download a Visual Studio extension which allows you to use a GUI to create a new model. It even runs your data through a bunch of machine learning algorithms and evaluates the most accurate one to use. Once your model is created, the Visual Studio extension generates the source code it used to create the model, so you can review exactly what it did and make adjustments where needed!
The model it creates can be easily retrained at any point. You can follow the Microsoft documentation here to retrain the model. All you need to do is load the model and pipeline that it previously used, then run a new set of data through them. It then saves the retrained model back to disk.
Just a note that I found by saving the pipeline .zip file to disk when initially creating the model made it easier later on when retraining.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With