Why wasn't PyPy included in standard Python?

Question

PyPy is not a fork of CPython, so it could never be merged directly into CPython.

Theoretically the Python community could universally adopt PyPy, PyPy could be made the reference implementation, and CPython could be discontinued. However, PyPy has its own weaknesses:

• CPython is easy to integrate with Python modules written in C, which is traditionally the way Python applications have handled CPU-intensive tasks (see for instance the SciPy project).
• The PyPy JIT compilation step itself costs CPU time -- it's only through repeated running of compiled code that it becomes faster overall. This means startup times can be higher, and therefore PyPy isn't necessarily as efficient for running glue code or trivial scripts.
• PyPy and CPython behavior is not identical in all respects, especially when it comes to "implementation details" (behavior that is not specified by the language but is still important at a practical level).
• CPython runs on more architectures than PyPy and has been successfully adapted to run in embedded architectures in ways that may be impractical for PyPy.
• CPython's reference counting scheme for memory management arguably has more predictable performance impacts than PyPy's various GC systems, although this isn't necessarily true of all "pure GC" strategies.
• PyPy does not yet fully support Python 3.x, although that is an active work item.

PyPy is a great project, but runtime speed on CPU-intensive tasks isn't everything, and in many applications it's the least of many concerns. For instance, Django can run on PyPy and that makes templating faster, but CPython's database drivers are faster than PyPy's; in the end, which implementation is more efficient depends on where the bottleneck in a given application is.

Another example: you'd think PyPy would be great for games, but most GC strategies like those used in PyPy cause noticeable jitter. For CPython, most of the CPU-intensive game stuff is offloaded to the PyGame library, which PyPy can't take advantage of since PyGame is primarily implemented as a C extension (though see: pygame-cffi). I still think PyPy can be a great platform for games, but I've never seen it actually used.

PyPy and CPython have radically different approaches to fundamental design questions and make different tradeoffs, so neither one is "better" than the other in every case.

For one, it's not 100% compatible with Python 2.x, and has only preliminary support for 3.x.

It's also not something that could be merged - The Python implementation that is provided by PyPy is generated using a framework they have created, which is extremely cool, but also completely disparate with the existing CPython implementation. It would have to be a complete replacement.

There are some very concrete differences between PyPy and CPython, a big one being how extension modules are supported - which, if you want to go beyond the standard library, is a big deal.

It's also worth noting that PyPy isn't universally faster.

See this video by Guido van Rossum. He talks about the same question you asked at 12 min 33 secs.

Highlights:

• lack of Python 3 compatibility
• lack of extension support
• not appropriate as glue code
• speed is not everything

After all, he's the one to decide...

One reason might be that according to PyPy site, it currently runs only on 32- and 64-bit Intel x86 architecture, while CPython runs on other platforms as well. This is probably due to platform-specific speed enhancements in PyPy. While speed is a good thing, people often want language implementations to be as "platform-independent" as possible.