apache-sparkmobius

Conver RDD to Dataset/DataFrame in Mobius


Is there an easy way to convert an RDD to a Dataset (or DataFrame) in Mobius. Basically something similar to the functionality provided by scala's

import sqlContext.implicits._

I know there's sqlContext.CreateDataFrame() but as far as I can tell that requires me to define my own StructType in order to do the conversion.


Solution

  • No. For now, sqlContext.CreateDataFrame is the only option. Feel free to create an issue in Mobius repo to get the discussion started if you think ToDF() is required on RDDs.