scalareflectionapache-flinkshapelesshlist

How to convert JSON to scala shapeless.hlist?


I got json like {"name":"susan","age":25},and a hint to json keyset like "name:String,age:Int",how to create a HList from that json?


Solution

  • Based on the code you added and then deleted, it seems, having a runtime-string hint "name:String,age:Int" and runtime-string json {"name":"susan","age":25}, you want to get an HList with runtime reflection. You can do this as follows

    import shapeless.HList
    import scala.reflect.runtime
    import scala.reflect.runtime.universe._
    import scala.tools.reflect.ToolBox
    val tb = runtime.currentMirror.mkToolBox()
    
    val jsonStr = """{"name":"susan","age":25}"""
    val hint = "name:String,age:Int"
    val classType = tb.define(tb.parse(s"case class Test($hint)").asInstanceOf[ImplDef]).asClass.toType
    val hlist = tb.eval(q"""
      import io.circe.generic.auto._
      import io.circe.parser.decode
      val classInstance = decode[$classType]($jsonStr)
    
      import shapeless.Generic
      Generic[$classType].to(classInstance.toOption.get)
    """).asInstanceOf[HList]
    println(hlist) // susan :: 25 :: HNil
    

    Please notice that you do everything at runtime so you'll have no access to type String :: Int :: HNil at compile time and hlist has static type just HList (not String :: Int :: HNil) and HList is actually not better than just List[Any].

    build.sbt

    libraryDependencies ++= Seq(
      scalaOrganization.value % "scala-reflect"  % scalaVersion.value,
      scalaOrganization.value % "scala-compiler" % scalaVersion.value,
      "com.chuusai" %% "shapeless" % "2.4.0-M1",
      "io.circe" %% "circe-core"    % "0.13.0",
      "io.circe" %% "circe-parser"  % "0.13.0",
      "io.circe" %% "circe-generic" % "0.13.0"
    )
    

    Actually, I guess we do someting strange. We use highly type-level libraries (shapeless, circe) aimed to static type safety and then run them at runtime with reflection ignoring type safety and getting actually List[Any] (HList).

    I guess that if List[Any] (list of field values) is enough for you then you just need to use a more runtime library. For example, with json4s

    import org.json4s.{JInt, JObject, JString, JValue}
    import org.json4s.jackson.JsonMethods._
    
    val jsonStr: String = """{"name":"susan","age":25}"""
    val json: JValue = parse(jsonStr) //JObject(List((name,JString(susan)), (age,JInt(25))))
    val l: List[JValue] = json.asInstanceOf[JObject].obj.map(_._2) //List(JString(susan), JInt(25))
    val res: List[Any] = l.map {
      case JString(s) => s
      case JInt(n)    => n
    } //List(susan, 25)
    

    build.sbt

    libraryDependencies += "org.json4s" %% "json4s-jackson" % "3.6.9"
    

    Actually the same can be done with Circe, just with parse instead of decode[A]

    import io.circe.{Json, JsonNumber}
    import io.circe.parser.parse
    
    val jsonStr: String = """{"name":"susan","age":25}"""
    val json: Json = parse(jsonStr).toOption.get //{"name":"susan","age":25}
    val l: List[Json] = json.asObject.get.values.toList //List("susan", 25)
    val res: List[Any] = l.map(_.fold[Any](null, null, (_: JsonNumber).toInt.get, identity[String], null, null)) //List(susan, 25)
    

    If you need an instance of case class or a tuple rather than HList replace

    tb.eval(q"""
      import io.circe.generic.auto._
      import io.circe.parser.decode
      val classInstance = decode[$classType]($jsonStr)
    
      import shapeless.Generic
      Generic[$classType].to(classInstance.toOption.get)
    """).asInstanceOf[HList] // susan :: 25 :: HNil
    

    with

    tb.eval(q"""
      import io.circe.generic.auto._
      import io.circe.parser.decode
      decode[$classType]($json).toOption.get
    """).asInstanceOf[Product] //Test(susan,25)
    

    or

    tb.eval(q"""
      import io.circe.generic.auto._
      import io.circe.parser.decode
      val classInstance = decode[$classType]($json)
    
      import shapeless.Generic
      Generic[$classType].to(classInstance.toOption.get).tupled
    """).asInstanceOf[Product] //(susan,25)
    

    correspondingly.