I've recently changed my program's directory layout: before, I had all my modules inside the "main" folder. Now, I've moved them into a directory named after the program, and placed an __init__.py
there to make a package.
Now I have a single .py file in my main directory that is used to launch my program, which is much neater.
Anyway, trying to load in pickled files from previous versions of my program is failing. I'm getting, "ImportError: No module named tools" - which I guess is because my module was previously in the main folder, and now it's in whyteboard.tools, not simply plain tools. However, the code that is importing in the tools module lives in the same directory as it, so I doubt there's a need to specify a package.
So, my program directory looks something like this:
whyteboard-0.39.4
-->whyteboard.py
-->README.txt
-->CHANGELOG.txt
---->whyteboard/
---->whyteboard/__init__.py
---->whyteboard/gui.py
---->whyteboard/tools.py
whyteboard.py launches a block of code from whyteboard/gui.py, that fires up the GUI. This pickling problem definitely wasn't happening before the directory re-organizing.
Pickle dump replaces current file data.
With pickle protocol v1, you cannot pickle open file objects, network connections, or database connections.
“Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream (from a binary file or bytes-like object) is converted back into an object hierarchy.
Python pickle module is used for serializing and de-serializing a Python object structure. Any object in Python can be pickled so that it can be saved on disk. What pickle does is that it “serializes” the object first before writing it to file. Pickling is a way to convert a python object (list, dict, etc.)
As pickle's docs say, in order to save and restore a class instance (actually a function, too), you must respect certain constraints:
pickle can save and restore class instances transparently, however the class definition must be importable and live in the same module as when the object was stored
whyteboard.tools
is not the "the same module as" tools
(even though it can be imported by import tools
by other modules in the same package, it ends up in sys.modules
as sys.modules['whyteboard.tools']
: this is absolutely crucial, otherwise the same module imported by one in the same package vs one in another package would end up with multiple and possibly conflicting entries!).
If your pickle files are in a good/advanced format (as opposed to the old ascii format that's the default only for compatibility reasons), migrating them once you perform such changes may in fact not be quite as trivial as "editing the file" (which is binary &c...!), despite what another answer suggests. I suggest that, instead, you make a little "pickle-migrating script": let it patch sys.modules
like this...:
import sys from whyteboard import tools sys.modules['tools'] = tools
and then cPickle.load
each file, del sys.modules['tools']
, and cPickle.dump
each loaded object back to file: that temporary extra entry in sys.modules
should let the pickles load successfully, then dumping them again should be using the right module-name for the instances' classes (removing that extra entry should make sure of that).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With