Unpickleable Data Types and dill

00:00 By now you should be starting to see that pickle can handle many of the data types that you might run into with your Python scripts. Unfortunately, it can’t handle everything. Certain objects like database connections, network sockets, running threads, and so on can’t be serialized using pickle.

00:17 But there might be a solution out there. In this lesson, you’re going to use the dill module to serialize a lambda function. dill is a third-party library that extends upon pickle.

00:28 It can work with functions that yield results, nested functions, and quite a few other cases where pickle doesn’t quite work on its own. To get started, I’ve created a new Python script called pickling_error.py.

00:41 Like before, go ahead and import pickle and then go ahead and define a lambda function, like square = lambda and then say x is x * x.

00:54 Now you can go ahead and say my_pickle and set this equal to pickle.dumps() and put the lambda function in there.

01:03 And then go ahead and print(my_pickle). Okay! Once you save this, go down to your terminal and you can try and run it.

01:15 You can see there was an issue here. When this tried to run, when pickle tried to dump that lambda expression into a string, you had this result here where there’s a pickling error and it can’t pickle this lambda function. So let’s try to get around this.

01:31 Inside your virtual environment, go pip install dill.

01:40 Back in the editor, you can go ahead and just replace pickle here with dill.

01:48 So now this case of pickle can be replaced with dill. And keep in mind that dill still has the same API as pickle, so you can continue to use the .dumps() method, which will take this square and serialize it into a byte string.

02:02 Save that and try and rerun it.

02:07 And look at that! You can see that it now printed out the byte string that represents that lambda. dill can serialize quite a few advanced data types, so if you feel the need to save the state of something that pickle isn’t working for, go ahead and try dill.

02:28 There are still some cases that dill can’t handle, so in the next lesson, you’re going to see how you can get around these strange cases by using the dunder methods .__getstate__() and .__setstate__().