spray json in scala: deserializing json with unknown fields without losing them

I'm looking for a solution to this but for json spray and my searches and attempts to get this working with json spray have failed thus far.

If I have the following json:

{
  "personalData": {
    "person": {
        "first": "first_name",
        "last": "last_name"
    },
    "contact": {
        "phone": "1111111",
        "address": {
            "line": "123 Main St",
            "city": "New York"
        }
    },
    "xx": "yy", // unknown in advanced
    "zz": { // unknown in advanced
        "aa": "aa",
        "bb": "bb",
        "cc": {
            "dd": "dd",
            "ee": "ee"
        }
    }
  }
}

We know for sure that the json will contain person and contact but we don't know what other fields may be created upstream from us that we don't care about/use.

I want to serialize this JSON into a case class containing person and contact but on the other hand, I don't want to lose the other fields (save them in a map so the class will be deserialized to the same json as received).

This is how far I've made it:

case class Address(
                    line:        String,
                    city:        String,
                    postalCode:  Option[String]
                  )

case class Contact(
                    phone:       String,
                    address:     Address
                  )

case class Person(
                   first:      String,
                   last:      String
                 )

case class PersonalData(
                         person:      Person,
                         contact:     Contact,
                         extra:       Map[String, JsValue]
                       )

implicit val personFormat = jsonFormat2(Person)
implicit val addressFormat = jsonFormat3(Address)
implicit val contactFormat = jsonFormat2(Contact)

implicit val personalDataFormat = new RootJsonFormat[PersonalData] {
  def write(personalData: PersonalData): JsValue = {
    JsObject(
      "person" -> personalData.person.toJson,
      "contact" -> personalData.contact.toJson,
      // NOT SURE HOW TO REPRESENT extra input
    )
  }

  def read(value: JsValue): CAERequestBEP = ???
}

Can someone help me do this with spray.json instead of play? I've spent such a long time trying to do this and can't seem to make it work.

Solution

In order to do that, you need to write your own formatter for PersonalDataFormat:

case class Person(first: String, last: String)
case class Address(line: String, city: String)
case class Contact(phone: String, address: Address)
case class PersonalData(person: Person, contact: Contact, extra: Map[String, JsValue])
case class Entity(personalData: PersonalData)

implicit val personFormat = jsonFormat2(Person)
implicit val addressFormat = jsonFormat2(Address)
implicit val contactFormat = jsonFormat2(Contact)
implicit object PersonalDataFormat extends RootJsonFormat[PersonalData] {
  override def read(json: JsValue): PersonalData = {
    val fields = json.asJsObject.fields
    val person = fields.get("person").map(_.convertTo[Person]).getOrElse(???) // Do error handling instead of ???
    val contact = fields.get("contact").map(_.convertTo[Contact]).getOrElse(???) // Do error handling instead of ???
    PersonalData(person, contact, fields - "person" - "contact")
  }

  override def write(personalData: PersonalData): JsValue = {
    JsObject(personalData.extra ++ ("person" -> personalData.person.toJson, "contact" -> personalData.contact.toJson))
  }
}

implicit val entityFormat = jsonFormat1(Entity)

val jsonResult = jsonString.parseJson.convertTo[Entity]

The result is:

Entity(PersonalData(Person(first_name,last_name),Contact(1111111,Address(123 Main St,New York)),Map(xx -> "yy", zz -> {"aa":"aa","bb":"bb","cc":{}})))

(Assuming the json is not exactly the json above, but a valid similar one)

Code run in Scastie