is python capable of running on multiple cores?

Question

Question: Because of python's use of "GIL" is python capable running its separate threads simultaneously?

---

Info:

After reading this I came away rather uncertain on whether or not python is capable of taking advantage of a multi-core processor. As well done as python is, it feels really weird to think that it would lack such a powerful ability. So feeling uncertain, I decided to ask here. If I write a program that is multi threaded, will it be capable of executing simultaneously on multiple cores?

Answer 1

The answer is "Yes, But..."

But cPython cannot when you are using regular threads for concurrency.

You can either use something like multiprocessing, celery or mpi4py to split the parallel work into another process;

Or you can use something like Jython or IronPython to use an alternative interpreter that doesn't have a GIL.

A softer solution is to use libraries that don't run afoul of the GIL for heavy CPU tasks, for instance numpy can do the heavy lifting while not retaining the GIL, so other python threads can proceed. You can also use the ctypes library in this way.

If you are not doing CPU bound work, you can ignore the GIL issue entirely (kind of) since python won't aquire the GIL while it's waiting for IO.

Answer 2

Python threads cannot take advantage of many cores. This is due to an internal implementation detail called the GIL (global interpreter lock) in the C implementation of python (cPython) which is almost certainly what you use.

The workaround is the multiprocessing module http://www.python.org/dev/peps/pep-0371/ which was developed for this purpose.

Documentation: http://docs.python.org/library/multiprocessing.html

(Or use a parallel language.)