What is the use of Python's basic optimizations mode? (python -O)

Question

Python has a flag -O that you can execute the interpreter with. The option will generate "optimized" bytecode (written to .pyo files), and given twice, it will discard docstrings. From Python's man page:

-O Turn on basic optimizations. This changes the filename extension for compiled (bytecode) files from .pyc to .pyo. Given twice, causes docstrings to be discarded.

This option's two major features as I see it are:

• Strip all assert statements. This trades defense against corrupt program state for speed. But don't you need a ton of assert statements for this to make a difference? Do you have any code where this is worthwhile (and sane?)

• Strip all docstrings. In what application is the memory usage so critical, that this is a win? Why not push everything into modules written in C?

What is the use of this option? Does it have a real-world value?

Answer 1

Another use for the -O flag is that the value of the __debug__ builtin variable is set to False.

So, basically, your code can have a lot of "debugging" paths like:

if __debug__:
     # output all your favourite debugging information
     # and then more

which, when running under -O, won't even be included as bytecode in the .pyo file; a poor man's C-ish #ifdef.

Remember that docstrings are being dropped only when the flag is -OO.

Answer 2

On stripping assert statements: this is a standard option in the C world, where many people believe part of the definition of ASSERT is that it doesn't run in production code. Whether stripping them out or not makes a difference depends less on how many asserts there are than on how much work those asserts do:

def foo(x):
    assert x in huge_global_computation_to_check_all_possible_x_values()
    # ok, go ahead and use x...

Most asserts are not like that, of course, but it's important to remember that you can do stuff like that.

As for stripping docstrings, it does seem like a quaint holdover from a simpler time, though I guess there are memory-constrained environments where it could make a difference.

Answer 3

If you have assertions in frequently called code (e.g. in an inner loop), stripping them can certainly make a difference. Extreme example:

$ python    -c 'import timeit;print timeit.repeat("assert True")'
[0.088717937469482422, 0.088625192642211914, 0.088654994964599609]
$ python -O -c 'import timeit;print timeit.repeat("assert True")'
[0.029736995697021484, 0.029587030410766602, 0.029623985290527344]

In real scenarios, savings will usually be much less.

Stripping the docstrings might reduce the size of your code, and hence your working set.

In many cases, the performance impact will be negligible, but as always with optimizations, the only way to be sure is to measure.

Answer 4

I have never encountered a good reason to use -O. I have always assumed its main purpose is in case at some point in the future some meaningful optimization is added.