AttributeError: 'module' object has no attribute 'func' Only With C++ Invocation

Question

I've been working on integrating a scripting subsystem into an existing architecture and have been employing the use of the boost.python library.

My goal is to have the python environment that I create to be identical, or as close to, that which is expected in existing code - i.e. preserving namespace conventions, etc. The trick here is that I have two source files illustrating a use case that works if I fire up the interpreter standalone but do not work in my environment, meaning I must be doing something wrong.

As a disclaimer I'm not an experienced python user so I'm bound to butcher the correct terminology here.

core.py

import sys

def func():    
    print "sys is",sys

print "call from core"  
func()

main.py

import core

print "---------GLOBALS-------------"
globals_ = dict(globals())
for g in globals_:
    print g

print "--------------LOCALS---------"   
locals_ = dict(locals())
for l in locals_:
    print l

print "call from main"
core.func()

Invoking python main.py from command prompt

call from core
sys is <module 'sys' (built-in)>
---------GLOBALS-------------
core
__builtins__
__name__
__file__
__doc__
__package__
--------------LOCALS---------
core
g
__builtins__
__file__
globals_
__package__
__name__
__doc__
call from main
sys is <module 'sys' (built-in)>

I understand the steps as being:

1. main invokes import to import core.py.
2. The core module is loaded; the core namespace and the attributes it provides become accessible from main. In the process, core invokes its func method, printing what is bound to sys.
3. main invokes the function bound under the module core.

Example of how I bring up the scripting environment:

void Run()
{
    Py_Initialize();
    if(PyErr_Occurred())
    {
        PyErr_Print();
    }

    // Install builtins in __main__
    boost::python::object main_module = boost::python::import(ToplevelScope.c_str());
    boost::python::object main_namespace = main_module.attr("__dict__");
    boost::python::dict main_dictionary = boost::python::extract<boost::python::dict>(main_namespace);
    main_dictionary["__builtins__"] =  boost::python::handle<>(boost::python::borrowed(PyEval_GetBuiltins()));

    // Load modules
    LoadModule("core.py",GetDataForCore());
    LoadModule("main.py",GetDataForMain());
}

void LoadModule(std::string name, char* data)
{
    // First, see if the module already exists.
    PyObject* new_module;
    if( new_module = PyImport_AddModule(name.c_str()) )
    {
        if( PyDict_GetItemString(new_module,"__name__") )
        {
            return; // already loaded. no work to do.
        }
    }

    // Initialize default namespace parameters - global, local
    boost::python::dict base_dict = boost::python::dict();
    base_dict["__builtins__"] = boost::python::handle<>(PyImport_AddModule("__builtin__"));
    base_dict["__name__"] = name.c_str();
    base_dict["__file__"] = name.c_str();
    base_dict["__doc__"] = "None";
    base_dict["__package__"] = name.c_str();

    boost::python::exec(    data,
                            boost::python::dict(base_dict),  // make a copy of base_dict for both global & local
                            boost::python::dict(base_dict) );
}

Resultant output:

call from core
sys is <module 'sys' (built-in)>
---------GLOBALS-------------
core
__builtins__
__name__
__file__
__doc__
__package__
--------------LOCALS---------
core
g
__builtins__
__file__
globals_
__package__
__name__
__doc__
call from main
[LOG] Framework error while loading script main.py.
Traceback (most recent call last):
  File "<string>", line 17, in <module>
AttributeError: 'module' object has no attribute 'func'

It appears that the function in core somehow isn't accessible to main even after an import.

I suspect this has to do with me not fully initializing the core module. Replacing exec with PyRun_String, PyImport_ImportModule... and they all yield the same result. Here I'm creating a copy of a base dictionary with name set to the module name (core in this case) and supplying it as the global & local dictionaries.

I think that I'm not fully initializing the module somehow and will be doing a lot more reading to put it together... Any insight will be greatly appreciated!!

Answer 1

It seems likes you're trying to reinvent the wheel. boost::python::import does exactly what you want to do. Loads a module, or retrieves it from sys.modules if it's already been imported before. If you don't want to use boost then you can use PyObject* PyImport_ImportModule(const char *name). Just make sure the module is on python's path.

However, if the module doesn't exist as a file (say the source only exists in memory), then the following will recreate that functionality (it doesn't do any error checking, you'll need to add that yourself).

py::object LoadModule(std::string name, const char* module_source)
{
    namespace py = boost::python;

    // create module name and filenames
    py::object name_str(py::str(name.c_str()));
    std::string file = name;
    file += ".py";
    py::object file_str(py::str(file.c_str()));

    // get helper modules
    py::object types(py::import("types"));
    py::object sys(py::import("sys"));

    if (py::dict(sys.attr("modules")).has_key(name_str)) {
        return sys.attr("modules")[name_str];
    }

    // create empty module
    py::object module(types.attr("ModuleType")(name_str));

    // init module dict
    py::dict base_dict = py::extract<py::dict>(module.attr("__dict__"));
    base_dict["__builtins__"] = py::import("__builtins__");
    base_dict["__name__"] = name_str;
    base_dict["__file__"] = file_str;

    // execute module code in module context
    py::exec(module_source, base_dict, base_dict);

    // add module to sys.modules
    sys.attr("modules")[name] = module;

    return module;
}