pythonapache-kafkafaust

Deserializing json object to Faust record where json has hyphenations in key names

I have a JSON object that looks like

{ 
  "timestamp": "2020-06-24T15:32:56.775518Z",
  "record-type": "data",
  "operation": "load",
  "partition-key-type": "primary-key",
  "schema-name": "legacy",
  "table-name": "test"
}

and I'm trying to deserialize the records into

class MetadataModel(faust.Record):
    timestamp: str
    record-type: str  # does not work, hyphens in name not allowed
    schema_name: str  # does not work
    tableName: str    # does not work

I'm probably missing something simple, but how do I go from a json object that has hyphenations in the keys to a python object. Any help would be much appreciated!

Solution

  • You can use field classes from faust.models.fields to provide custom names in JSON.

    import faust
from faust.models.fields import StringField


class MetadataModel(faust.Record):
    timestamp: str
    record_type: str = StringField(input_name='record-type')
    operation: str
    partition_key_type: str = StringField(input_name='partition-key-type')
    schema_name: str = StringField(input_name='schema-name')
    table_name: str = StringField(input_name='table-name')


json_bytes = b"""
    { 
      "timestamp": "2020-06-24T15:32:56.775518Z",
      "record-type": "data",
      "operation": "load",
      "partition-key-type": "primary-key",
      "schema-name": "legacy",
      "table-name": "test"
    }
"""

print(MetadataModel.loads(json_bytes, default_serializer='json'))

    Output:

    <MetadataModel: timestamp='2020-06-24T15:32:56.775518Z', record_type='data', operation='load', partition_key_type='primary-key', schema_name='legacy', table_name='test'>