rddspark-shell

RDD output in spark-shell differs from print(RDD) in idea

    val rddData1 = sc.makeRDD(1 to 10, 2)
    println(rddData1.glom.collect)

code in idea or spark-shell will output [[I@34a0ef00

but rddData1.glom.collect in spark-shell will output Array[Array[Int]] = Array(Array(1, 2, 3, 4, 5), Array(6, 7, 8, 9, 10))

how do i can get Array[Array[Int]] = Array(Array(1, 2, 3, 4, 5), Array(6, 7, 8, 9, 10)) in idea

Solution 
  • val conf = new SparkConf()
  .setMaster("local[*]")
  .setAppName("myAppName")
val sc = new SparkContext(conf)
val rddData1 = sc.makeRDD(1 to 10,2)
println(java.util.Arrays.toString( rddData1.collect()))
// output [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
// 结束任务
sc.stop()