Decoding Custom Types From JSON

When serialize/de-serialize heterogenous objects (i.e. Person, Student) how do I avoid having one encoder/decoder function? This will allow new encoder/decoder functions for new classes without changing the code of a router type decoder/encoder function.

Is it possible to bypass the default encoder option and directly have Python have the encoder function?

@techsukenik If you find the json module’s API inconvenient to work with, then you always have the option to use the adapter pattern by wrapping the deserialized dictionary with custom code at a higher level of abstraction:

person_dict = json.loads("""\
{
    "firstName": "Joe",
    "lastName": "Doe",
    "birthdate": "1978-05-12"
}
""")

person = Person(
    first_name=person_dict["firstName"],
    last_name=person_dict["lastName"],
    birth_date=date.fromisoformat(person_dict["birthdate"]))

Calling json.loads() returns a plain dictionary that you’re free to cherry-pick elements from and transform them as you wish. That way, you’ll have total control over what happens and when.

If you don’t mind keeping the serialization and deserialization logic together with your model class, then you might combine them:

import json
from datetime import date
from typing import NamedTuple

class Person(NamedTuple):
    first_name: str
    last_name: str
    birth_date: date

    @staticmethod
    def from_json(text: str) -> "Person":
        dict_ = json.loads(text)
        return Person(first_name=dict_["firstName"],
                      last_name=dict_["lastName"],
                      birth_date=date.fromisoformat(dict_["birthdate"]))

    def to_json(self) -> str:
        return json.dumps({
            "firstName": self.first_name,
            "lastName": self.last_name,
            "birthdate": self.birth_date.isoformat()
        })

Alternatively, you could come up with a more clever solution that would work universally across many model classes. For example, you’d store the information about a specific data type in JSON, and then use that information to pick the right deserializer like in the video.

@Bartosz Zaczyński …

For the JSON class it is possible to expand it to cover json.JSONEncoder and json.JSONDecoder?

See .. alexis-gomes19.medium.com/custom-json-encoder-with-python-f52c91b48cd2

Using the json.JSONEncoder, a universal ObjectEncoder can be coded that automatically support custom objects. stackoverflow.com/questions/3768895/how-to-make-a-class-json-serializable

import json
import inspect

class ObjectEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj, "to_json"):
            return self.default(obj.to_json())
        elif hasattr(obj, "__dict__"):
            d = dict(
                (key, value)
                for key, value in inspect.getmembers(obj)
                if not key.startswith("__")
                and not inspect.isabstract(value)
                and not inspect.isbuiltin(value)
                and not inspect.isfunction(value)
                and not inspect.isgenerator(value)
                and not inspect.isgeneratorfunction(value)
                and not inspect.ismethod(value)
                and not inspect.ismethoddescriptor(value)
                and not inspect.isroutine(value)
            )
            return self.default(d)
        return obj

Example

class C(object):
    c = "NO"
    def to_json(self):
        return {"c": "YES"}

class B(object):
    b = "B"
    i = "I"
    def __init__(self, y):
        self.y = y

    def f(self):
        print "f"

class A(B):
    a = "A"
    def __init__(self):
        self.b = [{"ab": B("y")}]
        self.c = C()

print json.dumps(A(), cls=ObjectEncoder, indent=2, sort_keys=True)

Result:

{
  "a": "A", 
  "b": [
    {
      "ab": {
        "b": "B", 
        "i": "I", 
        "y": "y"
      }
    }
  ], 
  "c": {
    "c": "YES"
  }, 
  "i": "I"
}

@techsukenik I took a stab at what I think you’re trying to achieve here and created a GitHub repository to showcase my approach. Hopefully, this will give you some inspiration or clear your doubts.

@Bartosz Zaczyński Thanks for spending the time writing the code. I look forward in doing a deep dive on the code.

Why did the last element of the output print as 36j? Where did the “j” character come from? at 3:56. The JSON only had two integers. Also, why did it print the two elements seperated by a “+” ? Thanks much!

@jtchestnut It’s the default string representation of complex numbers in Python. The letter “j” indicates the imaginary part of a complex number to avoid confusing it with regular addition:

>>> 42 + 36
78

>>> 42 + 36j
(42+36j)

>>> type(42 + 36j)
<class 'complex'>

Thank you for pointing me in the right direction here, Bartosz!

I believe I originally understood the types in the JSON, “real” and “imaginary” to be arbitrary labels from a random JSON data object (not translated into Python types). So, for a complex object, it sounds like the elements need to be labeld as Python types.

How would one take a random JSON data object from say, an AWS resource describe command, and load it into a Python object without having to modify the JSON file? I assume create a routine specific to process the JSON and map to a custom object? Was curious if there was some automated way to get close. I see object_hook() of json.load() may be a way.

@chestnutj2000 You’re welcome!

The “labels” or keys in JSON can indeed be arbitrary. It’s your custom decoder class that decides how to interpret them. In particular, to create an instance of a complex number, which isn’t normally JSON-serializable, you need two values that you can read from a JSON document once you know the expected keys or labels they’re stored at.

As to your second question, I’m not sure I’m following 🤔 If a piece of data was successfully serialized to JSON, then you can load it into Python using the json module. You’d only need a custom decoder if you wanted to use specific or non-standard representations of the serialized values.

I’m new to JSON. I found this tutorial confusing. It begins with a description of JSON which emphasizes its power as a data interchange format. Then it gave an interesting example of what you might do with it beyond the basic role of transmitting data. That opened my eyes. Where it lost me is when it showed a limitation of JSON and how you can get around it. Unfortunately, that meant the bulk of the tutorial was on overcoming a short coming. That introduced confusion, notice all the comments regarding complex number issues. Are we to conclude that to use JSON you should be ready to provide your own decoders? Or was that just muscle flexing and a neat solution? I would have liked to see more examples of where JSON was useful.