Named accumulator in pyspark

Question

In scala spark it is possible to create a named accumulator which will display in the web ui by

val accum = sc.accumulator(0, "My Accumulator")

However the pyspark sc.accumulator() method has no name argument. Is it possible to create a named accumulator in pyspark which will display in the web ui?

Answer 1

creating a named accumulator is not possible in pyspark. this issue has already been raised. you can track this thread https://issues.apache.org/jira/browse/SPARK-2868.