Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I know if pickle.dump() successfully saved the file

How can I know if pickle.dump() successfully saved the pickle file?
In the docs, I do not see a return value that indicates success or failure.

I'm working with python 3, and currently in a jupyter notebook

like image 557
SherylHohman Avatar asked Oct 18 '22 11:10

SherylHohman


1 Answers

If pickle.dump or pickle.dumps fails an error will be thrown. See the docs further down for what can and can't be pickled. You may also get an OSError (link) if some lower level system call fails

Note however that even if pickle.dump does not throw an error, you may still not be able to load the pickled data. It may for instance be the case that an object you pickled uses an import or references a function that was defined in the context of the pickling code, say a Jupyter notebook defines a custom function which is referenced by the pickled object. If you now ship that pickled object file to another machine it won't see the function that is referenced in the object and the unpickling will fail.

Similarly if there's an API change in a module that the picled object depends on, the import paths may have changed and the unpickling will again fail.

You may also want to have a look at dill which covers slightly more cases than pickle https://github.com/uqfoundation/dill

like image 149
Matti Lyra Avatar answered Oct 21 '22 08:10

Matti Lyra