Menu
I'd like to train a model using Spark ML Lib but then be able to export the model in a platform-agnostic format. Essentially I want to decouple how models are created and consumed.
My reason for wanting this decoupling is so that I can deploy a model in other projects. E.g.:
Has anyone done something like this with Spark ML Lib?
625
You can save your model by using the save method of mllib models. After storing it you can load it in another application. As @zero323 stated before, there is another way to achieve this, and is by using the Predictive Model Markup Language (PMML).
MLlib automated MLflow tracking is deprecated on clusters that run Databricks Runtime 10.1 ML and above, and it is disabled by default on clusters running Databricks Runtime 10.2 ML and above. Instead, use MLflow PySpark ML autologging by calling mlflow.
Version of Spark 1.4 now has support for this. See latest documentation. Not all models are available (see to be supported (see the JIRA issue SPARK-4587).
HTHs
199
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
