Encoding Custom Types to JSON

Working With JSON Data in Python Austin Cepalia 04:02

In this video, you’ll learn how to encode non-serializable types into JSON.

The dump() and dumps() methods allow us to include an optional argument default. Here, we can specify a custom function that will break our non-serializable type down into a serializable object containing the data that’s needed to re-construct it later.

Python
      
json_str = json.dumps(4+6j, default=complex_encoder)

Now we need to define complex_encoder(), which will convert our complex object into a tuple (which is serializable)

Python
      
    
def complex_encoder(z):
    if isinstance(z, complex):
        return (z.real, z.imag)
    else:
        type_name = z.__class__.__name__
        raise TypeError(f"Object of type {type_name} is not serializable")

We could also achieve this by subclassing json.JSONEncoder

Python
      
    
class ComplexEncoder(json.JSONEncoder):
    def default(self, z):
        if isinstance(z, complex):
            return (z.real, z.imag)
        else:
            return super().default(z)

If we choose this method instead, default wont work. We need to use cls instead.

Python
      
json_str = json.dumps(4+6j, cls=ComplexEncoder)

00:00 Welcome back to our series on working with JSON data in Python. In this video, we’re going to learn how we can encode the complex type into JSON format.

00:11 As usual, I’m here in Visual Studio Code, and I want to start by importing the json module. Now let’s create a new string called json_str and I’ll set it equal to json.dumps(), passing in the complex number 4 + 6j. And if I run this code here, you’ll see that the complex object is not serializable. Fortunately for us, the dump() and dumps() methods allow us to specify a second parameter called default. This gives us a chance to encode this object using a custom encoding function before the dump() function tries to do it itself.

00:53 I’ll type default = encode_complex and now we have to define that function. So up here, we will type def encode_complex(): and that will take a parameter of z. First, we want to make sure that this z object is actually a complex number, so we’ll say if isinstance(z, complex): then we want to return a tuple containing the real part of the z object as well as the imaginary part. Remember, the tuple is a serializable type, so we shouldn’t have any problems. Finally, we want to make sure that we’re not trying to encode any objects that are not of type complex, so we’ll say else: and we’ll obtain the type_name of whatever object was passed,

01:44 and finally, we’ll raise a TypeError with the f-string f"Object of type {type_name} is not JSON serializable". This way, if we try to serialize any other type, the dump() function will catch this error and it will know that it needs to use its default encoder instead of our custom function. So now, if I right-click here and I choose Run Code, we’ll see that we get a JSON string containing a list of the data needed to recreate our complex object later on. Using a custom encoding function is a good way of going about this, but it’s not the only way. We can achieve the same result by subclassing the JSONEncoder class. But if you hate object-oriented programming, I would suggest skipping the rest of this video and checking out my OOP course here on Real Python. But anyway, let’s see how this class works.

02:42 First, we’re going to move down below our encoding function and we’ll say class ComplexEncoder(), which will subclass json.JSONEncoder.

02:54 We need to override the .default() method, so we’ll type def default() with parameters of self and z.

03:02 Now, we’ll need to check to make sure that z is a complex object, just like before. And if it is, we’ll return a tuple with the real part and the imaginary part of the object packaged inside of it.

03:15 And if this is not a complex object, we’ll call the base class’s .default() method with self and z, just like this.

03:23 Just like before, this will tell the json.dumps() function to use the default encoder instead. Now, all we have to do is replace this default argument here with cls, and we’ll set that equal to our ComplexEncoder class.

03:39 And now if we run this, we’ll see that we get the same output as before. Whether you use the encoder class or the encoder function is really a matter of personal preference.

03:51 I usually go with the method because I find it easier, but it’s up to you. In the next video, we’ll learn how to decode our complex type.

Shay Elmualem on April 2, 2019

In the last snippet: Compex -> Complex, just in case of a copy-paste :)

Thanks.

Dan Bader RP Team on April 2, 2019

Ah, thanks Shay! The typo is fixed now :)

jianpingwu on July 27, 2019

It is very helpful, thanks! How to encode a list of custom objects?

alazejha on Dec. 30, 2020

When I write this code and run it, it saves the complex number as a string, not a complex number. Where’s my mistake? Thanks!

def complex_encoder(z):
    if isinstance(z,complex):
        return(z.real, z.imag)
    else:
        type_name = z.__class__.__name__
        raise TypeError(f"type {type_name} is not serializable")


json_str = json.dumps(4+6j, default= complex_encoder)
print(json_str)

with open("complex_data.json", "w") as write_file:
    json.dump(json_str, write_file)

print(type(json_str))

Bartosz Zaczyński RP Team on Jan. 4, 2021

@alazejha The value returned by your encoder function must be one of the data types supported in JSON, i.e.

| Python                                 | JSON   |
|----------------------------------------|--------|
| dict                                   | object |
| list, tuple                            | array  |
| str                                    | string |
| int, float, int- & float-derived Enums | number |
| True                                   | true   |
| False                                  | false  |
| None                                   | null   |

alazejha on May 17, 2021

Thank you, Bartosz, as I understand from your answer, the complex number will be saved as a string, and it’s OK. But when we deserialise this string back into Python, it will not return a complex number, unless we add ‘complex’ :true, and also ‘real’ and ‘imaginary’.

Does this mean that these has to be done manually?

Bartosz Zaczyński RP Team on May 18, 2021

@alazejha Here’s how you could serialize and deserialize your complex numbers:

import json

def serialize_complex(value: object):
    if isinstance(value, complex):
        return {"real": value.real, "imag": value.imag}

def deserialize_complex(value: dict):
    if "real" in value and "imag" in value:
        return complex(value["real"], value["imag"])
    return value

data = 4 + 6j

json_str = json.dumps(data, default=serialize_complex)
json_dict = json.loads(json_str, object_hook=deserialize_complex)

print(json_dict)

pauljohn32 on May 18, 2022

Why doesn’t your class have a return statement for giving back the super().... part, in line 16 of example.

I am writing to JSON from a huge dictionary and sometimes the data is not serializable, but I cannot predict if it will fail. So I just want to try/except, but don’t understand why you don’t return from else in your example.

My test

class JEncoder(json.JSONEncoder):
    def default(self, z):
        try:
            return super().default(self, z)
        except:
            return None
json.dump(X, cls=JEncoder)

Become a Member to join the conversation.