pythonpython-dataclassesmarshmallow

How to handle generic dataclasses in dataclasses-json?


I'm working on defining a schema for fetching data from an API using dataclasses-json, which is based on Marshmallow. A lot of lists in the schema are wrapped in a nodes field, like this:

{
  "foo": {
    "nodes": [
      {
        "bar": "baz"
      }
    ]
  }
}

My schema code currently looks something like this:

from dataclasses import dataclass, field
from typing import Generic, TypeVar

from dataclasses_json import dataclass_json

T = TypeVar("T")


@dataclass_json
@dataclass
class NodesWrapper(Generic[T]):
    nodes: list[T]

@dataclass_json
@dataclass
class Bar:
    bar: str

@dataclass_json
@dataclass
class Foo:
    foo: NodesWrapper[Bar]

When I run some actual data downloaded from the API (to avoid spamming it while I'm testing), I get something like this back in the REPL:

>>> from helpers.api_schemas.foobar import Foo
>>> with open(r'helpers\api_schemas\foobar_test_data.json') as f:
...     data = Foo.from_json(f.read())
...
>>> data
Foo(foo={'nodes': [{'bar': 'baz'}]})

What's supposed to be an instance of NodesWrapper is instead a dict!
Confused, I checked if there was something wrong with the Marshmallow schema:

>>> Foo.schema()
C:\Users\M1N3R\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\dataclasses_json\mm.py:263: UserWarning: Unknown type helpers.api_schemas.foobar.NodesWrapper[helpers.api_schemas.foobar.Bar] at Foo.foo: helpers.api_schemas.foobar.NodesWrapper[helpers.api_schemas.foobar.Bar] It's advised to pass the correct marshmallow type to `mm_field`.
  warnings.warn(
<FooSchema(many=False)>
>>>

A-ha! It wants a Marshmallow field type. Question is: what Marshmallow type does it want/need?


Solution

  • Yes, I know I'm answering my own question, but I have found a solution after an annoyingly long time, and I don't want any of you to find yourself stuck with the same predicament.

    I have moved from dataclasses-json to pydantic for handling ser/de in my app (I was working on a full re-write anyway, and it seems pydantic is even faster), and while it does have the same issue, I found a workaround through my ~~non~~impeccable Google-fu: Instead of using the subscripted generic type in the field (as in foo: NodesWrapper[Bar]), I subclassed the generic field, and redefined the nodes field to be a list of the target type, like:

    class Foo:
        class BarNodes(NodesWrapper[Bar]):
            nodes: list[Bar]
        foo: BarNodes
    

    A little VSCodium multi-cursor magic and testing later, all seems to work fine. There is no broken ser/de system in Ba Sing Se!