A pickle with jsonpickle (Python 3.7)

I have an issue with using jsonpickle. Rather, I believe it to be working correctly but it's not producing the output I want.

I have a class called 'Node'. In 'Node' are four ints (x, y, width, height) and a StringVar called 'NodeText'.

The problem with serialising a StringVar is that there's lots of information in there and for me it's just not necessary. I use it when the program's running, but for saving and loading it's not needed.

So I used a method to change out what jsonpickle saves, using the __getstate__ method for my Node. This way I can do this:

    def __getstate__(self):
        state = self.__dict__.copy()
        del state['NodeText']
        return state

This works well so far and NodeText isn't saved. The problem comes on a load. I load the file as normal into an object (in this case a list of nodes).

The problem loaded is this: the items loaded from json are not Nodes as defined in my class. They are almost the same (they have x, y, width and height) but because NodeText wasn't saved in the json file, these Node-like objects do not have that property. This then causes an error when I create a visual instance on screen of these Nodes because the StringVar is used for the tkinter Entry textvariable.

I would like to know if there is a way to load this 'almost node' into my actual Nodes. I could just copy every property one at a time into a new instance but this just seems like a bad way to do it.

I could also null the NodeText StringVar before saving (thus saving the space in the file) and then reinitialise it on loading. This would mean I'd have my full object, but somehow it seems like an awkward workaround.

If you're wondering just how much more information there is with the StringVar, my test json file has just two Nodes. Just saving the basic properties (x,y,width,height), the file is 1k. With each having a StringVar, that becomes 8k. I wouldn't care so much in the case of a small increase, but this is pretty huge.

Can I force the load to be to this Node type rather than just some new type that Python has created?

Edit: if you're wondering what the json looks like, take a look here:

{
  "1": {
    "py/object": "Node.Node",
    "py/state": {
      "ImageLocation": "",
      "TextBackup": "",
      "height": 200,
      "uID": 1,
      "width": 200,
      "xPos": 150,
      "yPos": 150
    }
  },
  "2": {
    "py/object": "Node.Node",
    "py/state": {
      "ImageLocation": "",
      "TextBackup": "",
      "height": 200,
      "uID": 2,
      "width": 100,
      "xPos": 50,
      "yPos": 450
    }
  }
}

Since the class name is there I assumed it would be an instantiation of the class. But when you load the file using jsonpickle, you get the dictionary and can inspect the loaded data and inspect each node. Neither node contains the property 'NodeText'. That is to say, it's not something with 'None' as the value - the attribute simple isn't there.

Solution

That's because jsonpickle doesn't know which fields are in your object normally, it restores only the fields passed from the state but the state doesn't field NodeText property. So it just misses it :)

You can add a __setstate__ magic method to achieve that property in your restored objects. This way you will be able to handle dumps with or without the property.

    def __setstate__(self, state):
        state.setdefault('NodeText', None)

        for k, v in state.items():
            setattr(self, k, v)

A small example

from pprint import pprint, pformat

import jsonpickle


class Node:
    def __init__(self) -> None:
        super().__init__()

        self.NodeText = Node
        self.ImageLocation = None
        self.TextBackup = None
        self.height = None
        self.uID = None
        self.width = None
        self.xPos = None
        self.yPos = None

    def __setstate__(self, state):
        state.setdefault('NodeText', None)

        for k, v in state.items():
            setattr(self, k, v)

    def __getstate__(self):
        state = self.__dict__.copy()

        del state['NodeText']
        return state

    def __repr__(self) -> str:
        return str(self.__dict__)


obj1 = Node()
obj1.NodeText = 'Some heavy description text'
obj1.ImageLocation = 'test ImageLocation'
obj1.TextBackup = 'test TextBackup'
obj1.height = 200
obj1.uID = 1
obj1.width = 200
obj1.xPos = 150
obj1.yPos = 150

print('Dumping ...')
dumped = jsonpickle.encode({1: obj1})
print(dumped)

print('Restoring object ...')
print(jsonpickle.decode(dumped))

outputs

# > python test.py
Dumping ...
{"1": {"py/object": "__main__.Node", "py/state": {"ImageLocation": "test ImageLocation", "TextBackup": "test TextBackup", "height": 200, "uID": 1, "width": 200, "xPos": 150, "yPos": 150}}}
Restoring object ...
{'1': {'ImageLocation': 'test ImageLocation', 'TextBackup': 'test TextBackup', 'height': 200, 'uID': 1, 'width': 200, 'xPos': 150, 'yPos': 150, 'NodeText': None}}