Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

loading modules by imp.load_source with same name resulting merger of the modules

Tags:

python

I would like to know if the following behavior is expected or a bug. I'm using CPython2.7

Create a file x.py

def funcA():
    print "funcA of x.py"
def funcB():
    print "funcB of x.py"

Create a file y.py

def funcB():
    print "funcB of y.py"

Create a file test.py

import sys, imp
# load x.py as fff
m = imp.load_source('fff', 'x.py')
print dir(m)
print sys.modules.get('fff')
# load y.py as fff
m = imp.load_source('fff', 'y.py')
print dir(m)    
print sys.modules.get('fff')

# import and exec func
import fff
fff.funcA()
fff.funcB()
print dir(fff)

The result

['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']
<module 'fff' from 'x.py'>
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']
<module 'fff' from 'y.py'>
funcA of x.py
funcB of y.py
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']

My expectation was that the 2nd imp.load_source would completely replace the module x.py with y.py. In fact sys.modules.get('fff') shows <module 'fff' from 'y.py'> but the resulting module was like a mix of x.py and y.py, and the latter has the precedence.

Is this expected or a bug?

EDIT: my test code had a typo. updated the result.

like image 207
Kenji Noguchi Avatar asked Feb 26 '13 06:02

Kenji Noguchi


1 Answers

This is an expected behavior.
See http://docs.python.org/2/library/imp.html

imp.load_source(name, pathname[, file])

Load and initialize a module implemented as a Python source file and return its module object. If the module was already initialized, it will be initialized again. The name argument is used to create or access a module object.

Since your second module has the same name as the first module, it won't replace the first one, but will be merged into the first one.

The source code gives us the same answer.
imp is a built-in module, defined in import.c.
Let's look at at the definition of load_source

static PyObject *
load_source_module(char *name, char *pathname, FILE *fp)
{
    ......
    m = PyImport_ExecCodeModuleEx(name, (PyObject *)co, pathname);
    Py_DECREF(co);

    return m;
}

It's just a wrapper for PyImport_ExecCodeModuleEx.

PyObject *
PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
{
    PyObject *modules = PyImport_GetModuleDict();
    PyObject *m, *d, *v;

    m = PyImport_AddModule(name);
    ......
    d = PyModule_GetDict(m);
    ......
    v = PyEval_EvalCode((PyCodeObject *)co, d, d);
    ......
}

Now, we just need to focus on PyImport_AddModule. Python uses it to obtain a module object. Your parsed source file will be put into this module object.

PyObject *
PyImport_AddModule(const char *name)
{
    PyObject *modules = PyImport_GetModuleDict();
    PyObject *m;

    if ((m = PyDict_GetItemString(modules, name)) != NULL &&
        PyModule_Check(m))
        return m;
    m = PyModule_New(name);
    if (m == NULL)
        return NULL;
    if (PyDict_SetItemString(modules, name, m) != 0) {
        Py_DECREF(m);
        return NULL;
    }
    Py_DECREF(m); /* Yes, it still exists, in modules! */

    return m;
}

Finally, we find the answer. Given a name, if some module already has this name, namely, name in sys.modules, Python won't create a new module, but will reuse that module.

like image 121
nymk Avatar answered Nov 11 '22 15:11

nymk