Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache Spark MLlib: How to import model from PMML

I have a PMML file which encodes a logistic regression model that was NOT exported from MLlib.

How can I import the model from PMML using MLlib in Java for evaluation/prediction?

(I know that MLlib can export to PMML, but I need to import from PMML)

like image 494
Qululu Avatar asked Jan 29 '17 11:01

Qululu


People also ask

How do I export a PMML model?

Right-click a model nugget on the models palette. (Alternatively, double-click a model nugget on the canvas and select the File menu.) On the menu, click Export PMML.

What is MLlib in Apache spark?

MLlib is Spark's machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. At a high level, it provides tools such as: ML Algorithms: common learning algorithms such as classification, regression, clustering, and collaborative filtering.

What is MLlib in Python?

MLlib is Spark's scalable machine learning library consisting of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as underlying optimization primitives, as outlined below: summary statistics. correlations.


1 Answers

You could use PMML4S-Spark to import PMML as a SparkML transformer, then make predictions/evaluations in Scala, for example:

import org.pmml4s.spark.ScoreModel

val model = ScoreModel.fromFile("the/pmml/model/path")
val scoreDf = model.transform(df)

If you use PySpark, you could use PyPMML-Spark, for example:

from pypmml_spark import ScoreModel

model = ScoreModel.fromFile('the/pmml/model/path')
score_df = model.transform(df)
like image 142
PredictFuture Avatar answered Oct 03 '22 15:10

PredictFuture