Should I optimise my python code like C++? Does it matter?

Question

I had an argument with a colleague about writing python efficiently. He claimed that though you are programming python you still have to optimise the little bits of your software as much as possible, as if you are writing an efficient algorithm in C++.

Things like:

• In an if statement with an or always put the condition most likely to fail first, so the second will not be checked.
• Use the most efficient functions for manipulating strings in common use. Not code that grinds strings, but simple things like doing joins and splits, and finding substrings.
• Call as less functions as possible, even if it comes on the expense of readability, because of the overhead this creates.

I say, that in most cases it doesn't matter. I should also say that context of the code is not a super-efficient NOC or missile-guidance systems. We're mostly writing tests in python.

What's your view of the matter?

Answer 1

My answer to that would be :

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

(Quoting Knuth, Donald. Structured Programming with go to Statements, ACM Journal Computing Surveys, Vol 6, No. 4, Dec. 1974. p.268)

If your application is doing anything like a query to the database, that one query will take more time than anything you can gain with those kind of small optimizations, anyway...

And if running after performances like that, why not code in assembly language, afterall ? Because Python is easier/faster to write and maintain ? Well, if so, you are right :-)

The most important thing is that your code is easy to maintain ; not a couple micro-seconds of CPU-time !
Well, maybe except if you have thousands of servers -- but is it your case ?

Answer 2

The answer is really simple :

• Follow Python best practices, not C++ best practices.
• Readability in Python is more important that speed.
• If performance becomes an issue, measure, then start optimizing.

Answer 3

This sort of premature micro-optimisation is usually a waste of time in my experience, even in C and C++. Write readable code first. If it's running too slowly, run it through a profiler, and if necessary, fix the hot-spots.

Fundamentally, you need to think about return on investment. Is it worth the extra effort in reading and maintaining "optimised" code for the couple of microseconds it saves you? In most cases it isn't.

(Also, compilers and runtimes are getting cleverer. Some micro-optimisations may become micro-pessimisations over time.)