I'm working on defining a schema for fetching data from an API using dataclasses-json, which is based on Marshmallow. A lot of lists in the schema are wrapped in a nodes
field, like this:
{
"foo": {
"nodes": [
{
"bar": "baz"
}
]
}
}
My schema code currently looks something like this:
from dataclasses import dataclass, field
from typing import Generic, TypeVar
from dataclasses_json import dataclass_json
T = TypeVar("T")
@dataclass_json
@dataclass
class NodesWrapper(Generic[T]):
nodes: list[T]
@dataclass_json
@dataclass
class Bar:
bar: str
@dataclass_json
@dataclass
class Foo:
foo: NodesWrapper[Bar]
When I run some actual data downloaded from the API (to avoid spamming it while I'm testing), I get something like this back in the REPL:
>>> from helpers.api_schemas.foobar import Foo
>>> with open(r'helpers\api_schemas\foobar_test_data.json') as f:
... data = Foo.from_json(f.read())
...
>>> data
Foo(foo={'nodes': [{'bar': 'baz'}]})
What's supposed to be an instance of NodesWrapper
is instead a dict
!
Confused, I checked if there was something wrong with the Marshmallow schema:
>>> Foo.schema()
C:\Users\M1N3R\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\dataclasses_json\mm.py:263: UserWarning: Unknown type helpers.api_schemas.foobar.NodesWrapper[helpers.api_schemas.foobar.Bar] at Foo.foo: helpers.api_schemas.foobar.NodesWrapper[helpers.api_schemas.foobar.Bar] It's advised to pass the correct marshmallow type to `mm_field`.
warnings.warn(
<FooSchema(many=False)>
>>>
A-ha! It wants a Marshmallow field type. Question is: what Marshmallow type does it want/need?
Yes, I know I'm answering my own question, but I have found a solution after an annoyingly long time, and I don't want any of you to find yourself stuck with the same predicament.
I have moved from dataclasses-json to pydantic for handling ser/de in my app (I was working on a full re-write anyway, and it seems pydantic is even faster), and while it does have the same issue, I found a workaround through my ~~non~~impeccable Google-fu: Instead of using the subscripted generic type in the field (as in foo: NodesWrapper[Bar]
), I subclassed the generic field, and redefined the nodes
field to be a list of the target type, like:
class Foo:
class BarNodes(NodesWrapper[Bar]):
nodes: list[Bar]
foo: BarNodes
A little VSCodium multi-cursor magic and testing later, all seems to work fine. There is no broken ser/de system in Ba Sing Se!