Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to do map and reduce in SparkR

How do I do map and reduce operations using SparkR? All I can find is stuff about SQL queries. Is there a way to do map and reduce using SQL?

like image 260
Matthew Jones Avatar asked Oct 31 '22 01:10

Matthew Jones


1 Answers

See Writing R data frames returned from SparkR:::map for an example (the question itself). In short, the blog post referred to by sph21 is out of date. As of the current date, both map and reduce have been hidden in SparkR as private methods - there are open tickets to resolve that issue.

like image 161
russellpierce Avatar answered Nov 15 '22 10:11

russellpierce