In the Spark GraphX javadoc the fromEdges()
method has undocumented arguments like evidence$17
. Presumably these are artifacts of the Scala implementation, but what should I do with them in Java?
public static <VD,ED> Graph<VD,ED> fromEdges(RDD<Edge<ED>> edges,
VD defaultValue,
StorageLevel edgeStorageLevel,
StorageLevel vertexStorageLevel,
scala.reflect.ClassTag<VD> evidence$16,
scala.reflect.ClassTag<ED> evidence$17)
UPDATE: found some other Scala/Java examples, and it seems like the right thing is ClassTag$.MODULE$.apply(myClass)
where myClass
is the class of the VD or ED type, respectively. Not that this actually works, as it leads to later, mysterious exceptions, like java.lang.ArrayStoreException: java.lang.Integer
deep in org.apache.spark.graphx.impl.EdgePartitionBuilder.toEdgePartition
UPDATE: Indeed, the ClassTag$.MODULE$.apply(Long.class) works for me. The error I was getting was due to passing in 1 for the default value instead of 1L.
The evidence arguments are filled in using ClassTag$.MODULE$.apply(myClass) where myClass is the class of the VD or ED type, respectively.
If you use Long.class for the ED and VD types, make sure to use a matching value for the defaults when you call Graph.fromEdges()
. In particular for Long.class use e.g. 1L
for the default value instead of 1
.