Pickling objects imported with importlib.util

Question

I ran into a problem while using Python's pickle. I need to load some Python modules by giving their file paths to importlib.util, like so:

import importlib.util
spec = importlib.util.spec_from_file_location('custom', 'C:\path\to\.py\file.py')
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

I would like to instantiate some objects from the loaded module and serialize them for later use, but when I try:

pickle.dump(module.ObjectFromModule(), open('C:\object\location\obj.p', 'wb'))

I get this: _pickle.PicklingError: Can't pickle : import of module 'custom' failed

If I try pickling the object that is imported via the import statement, this doesn't happen. How can I bypass this?

Answer 1

The easiest way to make it working is to manually add the module to the sys.module the following way:

import importlib.util
import sys

spec = importlib.util.spec_from_file_location('custom', 'C:\path\to\.py\file.py')
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
sys.modules['custom'] = module

Answer 2

Pickle depends on the module path. So, you should make sure that the module custom is in your sys.path.

In a similar application, I could make it work by doing something like:

file_path = 'C:\path\to\.py\file.py'
dir_name = os.path.dirname(file_path)
if dir_name not in sys.path:
    sys.path.append(dir_name)