In Python, why is a module implemented in C faster than a pure Python module, and how do I write one?

Question

The python documentation states, that the reason cPickle is faster than Pickle is, that the former is implemented in C. What does that mean exactly?

I am making a module for advanced mathematics in Python, and some calculations take a significant amount of time. Does that mean that if my program is implemented in C it can be made much faster?

I wish to import this module from other Python programs, just the way I can import cPickle.

Can you explain how to do implement a Python module in C?

Answer 1

When you write a function in python, a new function object is created, the function code is parsed and bytecompiled[and saved in the "func_code" attribute], so when you call that function the interpreter reads its bytecode and executes it.

If you write the same function in C, following C/Python API to make it avaiable in python, the interpreter will create the function object, but this function won't have a bytecode. When the interpreter finds a call to that function it calls the real C function, thus it executes at "machine" speed and not at "python-machine" speed.

You can verify this checking functions written in C:

>>> map.func_code
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'builtin_function_or_method' object has no attribute 'func_code'
>>> def mymap():pass
... 
>>> mymap.func_code
<code object mymap at 0xcfb5b0, file "<stdin>", line 1>

To understand how you can write C code for python use follow the guides in the official site.

Anyway, if you are simply doing N-dimensional array calculations numpy ought to be sufficient.