Python 2 and Python 3 dual development

Question

I'm just starting a new Python project, and ideally I'd like to offer Python 2 and 3 support from the start, with minimal developmental overhead. My question is, what is the best way of doing this for brand new projects?

I have come across projects that run 2to3, or even 3to2, as part of their installation script. This seems to be a very common way. However, there seems to be several different ways of doing this. I also came across Distribute.

There is also the option of trying to write polyglot Python 2/Python 3 code. Even though this seems like a horrible idea, I have noticed that I tend to write code lately that is more idiomatic as Python 3 code, even though I still run it as Python 2. I have a feeling this only helps my own transition when the day finally arrives, and doesn't do much for offering or at least helping dual support though.

Most of the projects offering dual support that I have seen added Python 3 support late, so I'm especially curious if there is a better way that is more suited for new projects, where you have the benefit of a clean slate.

Thanks!

Answer 1

In my experience, it depends on the kind of project.

If it is a library or very self contained application, a common choice is develop in Python 2.7 avoiding constructs deprecated in Python 3.x as much as possible and resort to automated tests to identify holes left by py2to3 that you will have to fix manually.

On the other side, for real life applications, be prepared to constantly stumble upon libraries that are not ported to py3k yet (sometimes important ones). Most of the time you will have no choice but port the library to Python 3, so if you can afford that, go for it. Usually I can't, that is why I'm not supporting Python 3 for this kind of project (but I struggle to write code that will be easier to port when opportune).

For unicode handling, I found this PyCon 2012 video very informative. The advice is good for both Python 2.x and 3.x: treat every string coming from outside as bytes and convert to unicode as soon as possible and output strings converting to bytes as late as possible. There is another very informative video about date/time handling.

[update]

This is an old answer. As of today (2019) there is no good rationale to start a project using Python 2.x and there are several compelling reasons to port older projects to Python 3.7+ and abandon support for Python 2.x.

Answer 2

In my experience it is better to not to use a library like six; I instead have a single compat.py for each package with just the needed code, not unlike Scott Griffiths's approach. six has also the burden of trying to support long-gone Python versions: the truth is that life is much easier when you accept that Pythons <=2.6 and <=3.2 are gone. In 2.7 there are backported compatibility features such as .view* methods on dicts that work exactly like their non-prefixed versions on Python 3; and Python 3.3 on the other hand supports u prefix on unicode strings again.

Even for very substantial packages, a compat.py module, which allows other code to work unchanged, can be quite short: here's an example from the pika package that my colleagues and I helped to make 2/3 polyglot. Pika is one of those projects that had really messed internals mixing unicode and 8 bit strings with each other, but now we've used it in production on Python 3 for well over 6 months without problems.

---

Other important thing is to always use the following __future__s when developing:

from __future__ import absolute_import, division, print_function 

I recommend against using unicode_literals, because there are some strings that need to be of the type called str on either platform. If you don't use unicode_literals, you can do the following:

• b'123' is the 8-bit string literal
• '123' is of the type str on both platforms
• u'123' is proper unicode text on both platforms

---

In any case, please do not do 2to3 on installation/package build time; some packages used to do that in the past - pip installing those packages took some seconds on Python 2, but closer to minute on Python 3.