apache-sparkscala-spark

Why we use Val for Accumulators and not Var in scala?


Why we use Val instead of Var for accumulators? If its like a counter that's shared across for multiple executor nodes to just update/change it, then it means reassigning a Val right?

val accum = sc.longAccumulator("New Accumulator")


Solution

  • In this case, we are instantiating a Class of LongAccumulator. Your logic is not wrong, but the object that we are storing in the accum is static, but inside this object, we have two var values. sum and count the object is immutable. But the values inside of it are not.

    You can see the details of the code here.