scalatype-conversionspark-streamingrdddstream

DStream to Rdd in Spark-Straming

I have a DStream[String,String] and I need to convert it to RDD[String,String]. Is there any way to do it? I need to do using Scala language.

Thanks in advance!!

Solution

  • A DStream is a discretized sequence of RDDs. Take a look to the direct stream API.

    Having your DStream,with the forEach function you can apply transformations for each RDD:

    val yourStream: DStream[String] = //...

yourStream.forEachRDD{ rdd =>
   // your rdd transformations...
}

    You can take a look to examples here