Menu
I am a beginner to the world of Machine Learning and the usage of Apache Spark.
I have followed the tutorial at https://databricks-training.s3.amazonaws.com/movie-recommendation-with-mllib.html#augmenting-matrix-factors, and was succesfully able to develop the application. Now, as it is required that today's web application need to be powered by real time recommendations, I would like my model to be ready for new data that keeps coming on the server.
The site has quoted:
A better way to get the recommendations for you is training a matrix factorization model first and then augmenting the model using your ratings.
How do I do that? I am using Python to develop my application. Also, please tell me how do I persist the model to use it again, or an idea how do I interface this with a web service. Thanking you
227
Alternating Least Square (ALS) is also a matrix factorization algorithm and it runs itself in a parallel fashion. ALS is implemented in Apache Spark ML and built for a larges-scale collaborative filtering problems.
regParam specifies the regularization parameter in ALS (defaults to 1.0). implicitPrefs specifies whether to use the explicit feedback ALS variant or one adapted for implicit feedback data (defaults to false which means using explicit feedback).
rank is the number of features to use (also referred to as the number of latent factors). iterations is the number of iterations of ALS to run. ALS typically converges to a reasonable solution in 20 iterations or less. lambda specifies the regularization parameter in ALS.
Also, the matrix factorization using Alternating Least Squares (ALS) algorithm which is a type of collaborative filtering is used to solve overfitting issues in sparse data and increases prediction ac-curacy. The overfitting problem arises in the data as the user-item rating matrix is sparse.
I don't think online learning is possible for ALS in Spark. That means you can't update the model while getting the data in real time. However, you can use the model to get the predictions.
Also, refer to: How to update Spark MatrixFactorizationModel for ALS
186
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
