Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Getting python -m module to work for a module implemented in C

Tags:

python

I have a pure C module for Python and I'd like to be able to invoke it using the python -m modulename approach. This works fine with modules implemented in Python and one obvious workaround is to add an extra file for that purpose. However I really want to keep things to my one single distributed binary and not add a second file just for this workaround.

I don't care how hacky the solution is.

If you do try to use a C module with -m then you get an error message No code object available for <modulename>.

like image 647
Roger Binns Avatar asked May 29 '11 03:05

Roger Binns


People also ask

How do I access a module written in Python from C?

You need include Python. h header file in your C source file, which gives you access to the internal Python API used to hook your module into the interpreter. Make sure to include Python. h before any other headers you might need.

Why won't Python recognize my module?

This is caused by the fact that the version of Python you're running your script with is not configured to search for modules where you've installed them. This happens when you use the wrong installation of pip to install packages.

Are Python modules written in C?

Most of the Python Libraries are written in the C programming language. The Python standard library consists of more than 200 core modules. All these work together to make Python a high-level programming language.


1 Answers

-m implementation is in runpy._run_module_as_main . Its essence is:

mod_name, loader, code, fname = _get_module_details(mod_name)
<...>
exec code in run_globals

A compiled module has no "code object" accociated with it so the 1st statement fails with ImportError("No code object available for <module>"). You need to extend runpy - specifically, _get_module_details - to make it work for a compiled module. I suggest returning a code object constructed from the aforementioned "import mod; mod.main()": (python 2.6.1)

    code = loader.get_code(mod_name)
    if code is None:
+       if loader.etc[2]==imp.C_EXTENSION:
+           code=compile("import %(mod)s; %(mod)s.main()"%{'mod':mod_name},"<extension loader wrapper>","exec")
+       else:
+           raise ImportError("No code object available for %s" % mod_name)
-       raise ImportError("No code object available for %s" % mod_name)
    filename = _get_filename(loader, mod_name)

(Update: fixed an error in format string)

Now...

C:\Documents and Settings\Пользователь>python -m pythoncom

C:\Documents and Settings\Пользователь>

This still won't work for builtin modules. Again, you'll need to invent some notion of "main code unit" for them.

Update:

I've looked through the internals called from _get_module_details and can say with confidence that they don't even attempt to retrieve a code object from a module of type other than imp.PY_SOURCE, imp.PY_COMPILED or imp.PKG_DIRECTORY . So you have to patch this machinery this way or another for -m to work. Python fails before retrieving anything from your module (it doesn't even check if the dll is a valid module) so you can't do anything by building it in a special way.

like image 135
ivan_pozdeev Avatar answered Oct 04 '22 21:10

ivan_pozdeev