Deserialize JSON distinguising missing and null values

I have a requirement to parse a JSON object, using play-json and distinguish between a missing value, a string value and a null value.

So for example I might want to deserialize into the following case class:

case class MyCaseClass(
  a: Option[Option[String]]
)

Where the values of 'a' mean:

None - "a" was missing - normal play-json behavipr
Some(Some(String)) - "a" had a string value
Some(None) - "a" had a null value

So examples of the expected behavior are:

{}

should deserialize to myCaseClass(None)

{
  "a": null
} 

should deserialize as myCaseClass(Some(None))

{
  "a": "a"
}

should deserialize as myCaseClass(Some(Some("a"))

I've tried writing custom formatters, but the formatNullable and formatNullableWithDefault methods don't distinguish between a missing and null value, so the code I've written below cannot generate the Some(None) result

object myCaseClass {
  implicit val aFormat: Format[Option[String]] = new Format[Option[String]] {
    override def reads(json: JsValue): JsResult[Option[String]] = {
      json match {
        case JsNull => JsSuccess(None) // this is never reached
        case JsString(value) => JsSuccess(Some(value))
        case _ => throw new RuntimeException("unexpected type")
      }
    }
    override def writes(codename: Option[String]): JsValue = {
      codename match {
        case None => JsNull
        case Some(value) =>  JsString(value)
      }
    }
  }

  implicit val format = (
      (__ \ "a").formatNullableWithDefault[Option[String]](None)
  )(MyCaseClass.apply, unlift(MyCaseClass.unapply))
}

Am I missing a trick here? How should I go about this? I am very much willing to encode the final value in some other way than an Option[Option[Sting]] for example some sort of case class that encapsulates this:

case class MyContainer(newValue: Option[String], wasProvided: Boolean)

Solution

I recently found a reasonable way to do this. I'm using Play 2.6.11 but I'm guessing the approach will transfer to other recent versions.

The following snippet adds three extension methods to JsPath, to read/write/format fields of type Option[Option[A]]. In each case a missing field maps to a None, a null to a Some(None), and a non-null value to a Some(Some(a)) as the original poster requested:

import play.api.libs.json._

object tristate {
  implicit class TriStateNullableJsPathOps(path: JsPath) {
    def readTriStateNullable[A: Reads]: Reads[Option[Option[A]]] =
      Reads[Option[Option[A]]] { value =>
        value.validate[JsObject].flatMap { obj =>
          path.asSingleJsResult(obj) match {
            case JsError(_)           => JsSuccess(Option.empty[Option[A]])
            case JsSuccess(JsNull, _) => JsSuccess(Option(Option.empty[A]))
            case JsSuccess(json, _)   => json.validate[A]
                                             .repath(path)
                                             .map(a => Option(Option(a)))
          }
        }
      }

    def writeTriStateNullable[A: Writes]: OWrites[Option[Option[A]]] =
      path.writeNullable(Writes.optionWithNull[A])

    def formatTriStateNullable[A: Format]: OFormat[Option[Option[A]]] =
      OFormat(readTriStateNullable[A], writeTriStateNullable[A])
  }
}

Like previous suggestions in this thread, this method requires you to write out a JSON format in full using the applicative DSL. It's unfortunately incompatible with the Json.format macro, but it gets you close to what you want. Here's a use case:

import play.api.libs.json._
import play.api.libs.functional.syntax._
import tristate._

case class Coord(col: Option[Option[String]], row: Option[Option[Int]])

implicit val format: OFormat[Coord] = (
  (__ \ "col").formatTriStateNullable[String] ~
  (__ \ "row").formatTriStateNullable[Int]
)(Coord.apply, unlift(Coord.unapply))

Some examples of writing:

format.writes(Coord(None, None))
// => {}

format.writes(Coord(Some(None), Some(None)))
// => { "col": null, "row": null }

format.writes(Coord(Some(Some("A")), Some(Some(1))))
// => { "col": "A", "row": 1 }

And some examples of reading:

Json.obj().as[Coord]
// => Coord(None, None)

Json.obj(
  "col" -> JsNull, 
  "row" -> JsNull
).as[Coord]
// => Coord(Some(None), Some(None))

Json.obj(
  "col" -> "A", 
  "row" -> 1
).as[Coord]
// => Coord(Some(Some("A")), Some(Some(1)))

As a bonus exercise for the reader, you could probably combine this with a little shapeless to automatically derive codecs and replace the Json.format macro with a different one-liner (albeit one that takes longer to compile).